Generating Sentences by Editing Prototypes
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints.”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Guu et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prompt augmentation is closely related to retrieval-based methods that provide more textual context to the model to improve performance [37], a method that has also been shown to be effective in prompt-based learning [99].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 3×
“Interestingly, the top-10 performance of our model in Table 5 is nearly as good as the performance of GloVe vectors on the simpler lexical analogy task, despite the fact that the sentence prediction task is harder.”
MethodInfluentialEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“Our use of lexical similarity neighborhoods is comparable to context windows used in word vector training (Mikolov et al., 2013a).”
BackgroundMethodAuto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 6×
“aining p(xjx0) as a conditional variational autoencoder (C-VAE) (it is conditioned on x0). We maximize the objective via SGD, approximating the gradient using the usual Monte Carlo estimate for VAEs (Kingma and Welling, 2014). Neural editor p edit(xjx0;z): We implement our neural editor as a left-to-right sequence-to-…”
MethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“Neural language models (Bengio et al., 2003) based upon recurrent neural networks and sequence-to-sequence architectures (Sutskever et al., 2014) have been widely used due to their flexibility and performance across a wide-range of NLP tasks (Ju-rafsky and Martin, 2000; Kalchbrenner and Blun-som,…”
BackgroundMethodA Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585 · cited 1×
“More generally, results in manifold learning demonstrate that a weak metric such as lexical similarity can be used to extract semantic similarity through distributional statistics (Tenenbaum et al., 2000; Hashimoto et al., 2016).”
BackgroundOn Estimation of a Probability Density Function and Mode
Parzen, E. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704472 · Cited by 10,585 · cited 1×
“From a generative modeling perspective, editing randomly sampled training sentences closely resembles nonparametric kernel density estimation (Parzen, 1962) where one samples points from a training set, and adds noise to smooth the density.”
BackgroundGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 2×
“Our work is motivated by an emerging consensus that attention-based mechanisms (Bahdanau et al., 2015) can substantially improve performance on various sequence to sequence tasks by capturing more information from the input sequence (Wu et al., 2016; Vaswani et al., 2017).”
BackgroundMethodAn Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361
Research works printed but not linked: 1
- Bengio Y., 2003, Journal of machine learning research, 3, 1137
Cited by
Items 201–293 of 293
A Submodular Optimization Framework for Imbalanced Text Classification With Data Augmentation
Alemayehu, E., & Fang, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3267669 · Cited by 4
“Guu et al. [27] use a VAE that encodes an item and an offset vector to generate an item whose latent representation is closest to the combination of the latent representation of the input item and the offset vector.”
BackgroundGârbacea, C., Carton, S., Yan, S., & Mei, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00398 · Cited by 4
Learning to Reuse Translations: Guiding Neural Machine Translation with ExamplesPreprint
Cao, Q., Kuang, S., & Xiong, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10732 · Cited by 4
“Guu et al. [11] present a new generative language model for sentences that first samples a prototype sentence and then edits it into a new sentence.”
Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models
Doostmohammadi, E., Norlund, T., Kuhlmann, M., & Johansson, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.45 · Cited by 12
Automatic Document Sketching: Generating Drafts from Analogous Texts
Wu, Z., Galley, M., Brockett, C., Zhang, Y., & Dolan, B. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.185 · Cited by 9
An Idiom Reading Comprehension Model Based on Multi-Granularity Reasoning and Paraphrase Expansion
Dai, Y., Liu, Y., Yang, L., & Fu, Y. (2023) · Applied Sciences · doi:10.3390/app13095777 · Cited by 3
GRACE: Gradient-guided Controllable Retrieval for Augmenting Attribute-based Text Generation
Wen, Z., Tian, Z., Huang, Z., Yang, Y.-X., Jian, Z.-X., Wang, C.-J., & Li, D.-S. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2023.findings-acl.530 · Cited by 6
“Another line of studies explicitly extracts a skeleton from the retrieved data and trains the model to complete or revise it (Guu et al., 2018; Cai et al., 2019a,b).”
BackgroundAudience-Centric Natural Language Generation via Style Infusion
Moorjani, S., Krishnan, A., Sundaram, H., Masłowska, E., & Sankar, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.138 · Cited by 5
Towards Example-Based NMT with Multi-Levenshtein Transformers
Bouthors, M., Crego, J., & Yvon, F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.113 · Cited by 12
Deliberation Learning for Image-to-Image Translation
He, T., Xia, Y., Lin, J., Tan, X., He, D., Qin, T., & Chen, Z. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/345 · Cited by 3
Learning to Flip the Sentiment of Reviews from Non-Parallel Corpora
Kruengkrai, C. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1659 · Cited by 5
“Our work is closely related to classical approaches for discovering parallel sentences in non-parallel corpora (Fung and Cheung, 2004; Munteanu and Marcu, 2005) and prototype-then-edit approaches (Guu et al., 2018).”
BackgroundMethodLearning Literary Style End-to-end with Artificial Neural Networks
Yamshchikov, I. P., & Tikhonov, A. (2019) · Advances in Science Technology and Engineering Systems Journal · doi:10.25046/aj040614 · Cited by 4
“Yet the problem of style transfer differs from the stylized text generation significantly: a human-written input could be used to control the saliency of the output [22].”
BackgroundNVSRN: A Neural Variational Scaling Reasoning Network for Initiative Response Generation
Chang, J., He, R., Xu, H., Han, K., Wang, L., Li, X., & Dang, J. (2019) · Proceedings · doi:10.1109/icdm.2019.00015 · Cited by 3
“It can be easily extended to other tasks, such as style transfer [23], [24], sentence editing [25] and so on.”
BackgroundMethodUnderstanding and Improving the Exemplar-based Generation for Open-domain Conversation
Han, S., Kim, B., Seo, S., Erdenee, E., & Chang, B. (2022) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2022.nlp4convai-1.18 · Cited by 3
Enhancing Text Generation via Parse Tree Embedding
Duan, D., Zhang, Q., Han, Z., & Xiong, H. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/4096383 · Cited by 3
“A new generative model of sentences was addressed, which first sampled a prototype sentence from the training corpus and then edited it into a new sentence and improved perplexity on language modeling and generated higher-quality outputs according to human evaluation [31].”
Shi, Z., Liu, H., Min, M. R., Malon, C. D., Li, L. E., & Zhu, X. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.171 · Cited by 9
“To remedy the problem, we propose to leverage prototype-based generation approaches (Guu et al., 2018; Hashimoto et al., 2018) which can explicitly expose concepts of other training examples by asking the model to decide what prototypes to retrieve in either a heuristic or learned way.”
MethodTopic-word-constrained sentence generation with variational autoencoder
Song, T., Sun, J., Liu, X., Song, J., & Peng, W. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.06.016 · Cited by 3
Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control
Fan, X., Lyu, Y., Liang, P. P., Salakhutdinov, R., & Morency, L.-P. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.758 · Cited by 3
“…1987; Xu et al., 2012), neural generative models (Prabhumoye et al., 2018; Lample et al., 2019; He et al., 2020), Retrieve-and-Edit approaches (Li et al., 2018; Hashimoto et al., 2018; Guu et al., 2018; Sudhakar et al., 2019; Madaan et al., 2020), and Transformer-based approach (Lyu et al., 2021)).”
MethodSelf-citationAn Instance Level Approach for Shallow Semantic Parsing in Scientific Procedural Text
Swarup, D., Bajaj, A., Mysore, S., O’Gorman, T., Das, R., & McCallum, A. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.270 · Cited by 3
“a dynamic retrieval model trained along side the label prediction models similar to Guu et al. (2020), we expect this would be feasible particularly given the small dataset sizes in this domain.”
MethodPun Generation with SurprisePreprint
He, H., Peng, N., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06828 · Cited by 10
“We then develop an unsupervised approach to generate puns based on a retrieve-and-edit framework (Guu et al., 2018; Hashimoto et al., 2018) given an unhumorous corpus (Figure 2).”
MethodSelf-citationExtract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine TranslationPreprint
Wu, J., Wang, X. E., & Wang, W. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02331 · Cited by 9
“Unlike other studies using the editing to generate more structural sentences (Guu et al., 2018; Hashimoto et al., 2018), here the revised sentences are designed to serve as better pivotal points in the target language space to guide the translation procedure.”
Text Generation with Exemplar-based Adaptive DecodingPreprint
Peng, H., Parikh, A. P., Faruqui, M., Dhingra, B., & Das, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04428 · Cited by 13
“Without committing to an explicit slot-filling process, a typical method is to include ex-emplars as additional inputs to the sequence-to-sequence models (Gu et al., 2018; Pandey et al., 2018; Guu et al., 2018, inter alia ).”
InfluentialTowards Minimal Supervision BERT-based Grammar Error CorrectionPreprint
Li, Y., Anastasopoulos, A., & Black, A. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03521 · Cited by 7
“A retrieve-edit model is proposed for text generation (Guu et al. 2017).”
MethodControllable Paraphrasing and Translation with a Syntactic Exemplar.Preprint
Chen, M., Wiseman, S., & Gimpel, K. (2020) · arXiv (Cornell University) · Cited by 3
Few-shot Controllable Style Transfer for Low-Resource Settings: A Study in Indian Languages.Preprint
Krishna, K., Nathani, D., García, X., Samanta, B., & Talukdar, P. P. (2021) · arXiv (Cornell University) · Cited by 3
SemStamp: A semantic watermark with paraphrastic robustness for text generation
Hou, A. B., Zhang, J., He, T., Wang, Y., Chuang, Y.-S., Wang, H., Shen, L., Van Durme, B., Khashabi, D., & Tsvetkov, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.03991 · Cited by 3
Learning from masked analogies between sentences at multiple levels of formality
Wang, L., & Lepage, Y. (2023) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-023-09918-2 · Cited by 3
“In case of analogies beyond words, most research has been restricted to the formal level like in [2, 10] or a near formal level involving one-word semantics like in [11, 12].”
Meeting the 2020 Duolingo Challenge on a Shoestring
Nomoto, T. (2020) · Workshop on Neural Generation and Translation (WNGT) · doi:10.18653/v1/2020.ngt-1.14 · Cited by 2
“Following (Guu et al., 2018), we applied a Gaussian noise on the output of the encoder as it was sent to the decoder (Fig.”
BackgroundMethodExploring Methods for Generating Feedback Comments for Writing Learning
Hanawa, K. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.270 · Cited by 2
Effective Unsupervised Constrained Text Generation based on Perturbed Masking
Fu, Y., Ou, W., Zhou, Y., & Lin, Y. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.111 · Cited by 4
“A new sentence generation model is proposed by (Guu et al., 2018), where a prototype sentence is first extracted from the training corpus and then edited into a new sentence.”
BackgroundData Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase
Yerukola, A., Bretan, M., & Jin, H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.159 · Cited by 4
Controlling Text Edition by Changing Answers of Specific Questions
Sha, L., Hohenecker, P., & Lukasiewicz, T. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.110 · Cited by 2
Exemplar VAEs for Exemplar based Generation and Data AugmentationPreprint
Norouzi, S., Fleet, D. J., & Norouzi, M. (2020) · arXiv (Cornell University) · Cited by 2
Variational Inference for Learning Representations of Natural Language EditsPreprint
Marrese-Taylor, E., Reid, M., & Matsuo, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.09143 · Cited by 2
“Secondly, we consider the approach by Guu et al. (2018). To adapt this to our setting, we skip the sampling procedure required in their case since our edits are already pairs of sentences and proceed to directly incorporate their edit encoding mechanism into our model.”
BackgroundMethodInfluentialImproving Text Generation with Student-Forcing Optimal TransportPreprint
Wang, G., Li, C., Li, J., Fu, H., Lin, Y.-C., Chen, L., Zhang, Y., Tao, C., Zhang, R., Wang, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05994 · Cited by 2
“Jaccard distance has been used to retrieve prototypes for sentence generation (Guu et al., 2018).”
MethodExemplar-Controllable Paraphrasing and Translation using BitextPreprint
Chen, M., Wiseman, S., & Gimpel, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05856 · Cited by 3
“There is a long history of example-based machine translation ( Nagao, 1984; Somers, 1999), and recently researchers find it helpful to build generations upon sentential exemplars ( Guu et al., 2018; Weston et al., 2018).”
BackgroundA Simple Survey of Pre-trained Language ModelsPreprint
Zhu, Z. (2022) · Preprints.org · doi:10.20944/preprints202208.0238.v1 · Cited by 4
Analyzing Methods for Generating Feedback Comments for Language Learners
Hanawa, K., Nagata, R., & Inui, K. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.901 · Cited by 1
Wang, D., Wang, L., Tang, K., Bo, Q., & Han, B. (2024) · IEEE Access · doi:10.1109/access.2024.3516088 · Cited by 2
“In this paper, along the lines of Guu [26] and others,we adopt the word vectors of different words in the original sentence x and the reference paraphrase sentence y as the edit vectors from the original sentence x to the reference paraphrase sentence y .”
Preventing Posterior Collapse with DVAE for Text Modeling
Song, T., Huang, Z., Liu, X., & Sun, J. (2025) · Entropy · doi:10.3390/e27040423 · Cited by 2
“Some methods use other distributions as the prior instead of Gaussian, such as the von Mises–Fisher distribution [22,23] and the uniform distribution [24,25], by which the KL regularization term is independent of the data instances.”
Guo, D., Luo, Z., Bouguila, N., & Fan, W. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356888 · Cited by 1
Paraphrase Generation: A Review from RNN to Transformer based Approaches
Singh, A., & Josan, G. S. (2022) · INTERNATIONAL JOURNAL OF NEXT-GENERATION COMPUTING · doi:10.47164/ijngc.v13i1.377 · Cited by 3
Generative Sentiment Transfer via Adaptive Masking
Xie, Y., Xu, J., Qiao, L., Liu, Y., Huang, F., & Li, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33383-5_16 · Cited by 1
Doolittle: Benchmarks and Corpora for Academic Writing Formalization
Diao, S., Lei, Y., Pan, L., Fang, T., Zhou, W., Keh, S. S., Kan, M., & Zhang, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.809 · Cited by 5
“Post-editing (No-vak et al., 2016; Xia et al., 2017; Guu et al., 2018; Freitag et al., 2019) is designed to rectify typical errors in machine translation, thereby augmenting the generation quality, as measured by BLEU.”
MethodUnsupervised Disentanglement Learning Model for Exemplar-Guided Paraphrase Generation
Li, L., Cai, Y., & Wu, X. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3374124 · Cited by 1
“Following the approach of Chen et al. [6], the semantic variable Y is assumed to follow the von Mises-Fisher (vMF) distribution, which has been used to model sentence similarity [16], and the prior is assumed to be vMF ( · , 0) .”
MethodA Latent Variable Model with Hierarchical Structure and GPT-2 for Long Text Generation
Zhao, K., Ding, H., Ye, K., Cui, X., & Fu, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86383-8_24 · Cited by 1
Incorporating Pseudo-Parallel Data for Quantifiable Sequence Editing.Preprint
Liao, Y., Bing, L., Li, P., Shi, S., Lam, W. P., & Zhang, T. (2018) · arXiv (Cornell University) · Cited by 1
Siamese Networks for Semantic Pattern SimilarityPreprint
Benajiba, Y., Sun, J., Zhang, Y., Jiang, L., Weng, Z., & Biran, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.06604 · Cited by 1
“[17] and [18] generate novel sentences by performing edits on another sentence, which tend to preserve the original semantic pattern.”
BackgroundALTER: Auxiliary Text Rewriting Tool for Natural Language GenerationPreprint
Xu, Q., Xu, C., & Qu, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06564 · Cited by 3
“Recently, there has been a series of work on conducting edit operations on text to advance automatic natural language generation (Guu et al., 2018; Li et al., 2018).”
BackgroundGenerating (Formulaic) Text by Splicing Together Nearest NeighborsPreprint
Wiseman, S., Bačkurs, A., & Stratos, K. (2021) · arXiv (Cornell University) · Cited by 1
Generating Related WorkPreprint
Shah, D., & Barzilay, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08668 · Cited by 4
“…Several tasks such as article generation (Liu et al., 2018), dialogue (Moghe et al., 2018; Weston et al., 2018; Di-nan et al., 2018; Fan et al., 2020), translation (Gu et al., 2018) and language modeling (Guu et al., 2018; Khandelwal et al., 2019) can be categorized as query driven generation.”
BackgroundBeyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic ModelsPreprint
Johnson, D. D., Austin, J., van den Berg, R., & Tarlow, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07675 · Cited by 15
“A number of other edit-based generative models have been proposed, including Guu et al. (2018) which edits prototypical examples in a latent space.”
BackgroundDeep learning-based text augmentation for named entity recognition
Surana, T. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/171105 · Cited by 1
Generating Contradictory, Neutral, and Entailing SentencesPreprint
Shen, Y., Tan, S., Huang, C.-W., & Courville, A. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.02710 · Cited by 3
“Both Gupta et al. (2017) and Guu et al. (2017) introduce models of the form p ( x | z , x (cid:48) ) , where x is a paraphrase of x (cid:48) , and z represents the variability in the output sentence.”
BackgroundEnriching Article Recommendation with Phrase AwarenessPreprint
Chen, C., Chou, S.-C., & Ku, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.01808
“Learning from humans has shown to be effective for designing better AI systems [9].”
BackgroundComposition and decomposition of GANsPreprint
Harn, Y., Chen, Z., & Jojic, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07667
“Following Guu et al. (2017) we use Jaccard distance to find nearest neighbor sentences.”
QuaSE: Accurate Text Style Transfer under Quantifiable GuidancePreprint
Liao, Y., Bing, L., Li, P., Shi, S., Lam, W. P., & Zhang, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07007 · Cited by 1
“Another line of works conduct editing on an existing sentence and expect that the output will serve particular purposes better (Guu et al., 2018).”
BackgroundMethodInfluentialLearning to Decipher Hate SymbolsPreprint
Qian, J., ElSherief, M., Belding, E., & Wang, W. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02418
“For example, Zhao et al. (2017) apply CVAE to dialog generation, while Guu et al. (2018) use CVAE for sentence generation.”
MethodA Multi-Task Approach for Disentangling Syntax and Semantics in Sentence RepresentationsPreprint
Chen, M., Tang, Q., Wiseman, S., & Gimpel, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01173 · Cited by 11
“…as a concentration parameter analogous to the variance in a Gaussian distribution. vMF has been used for modeling similarity between two sentences (Guu et al., 2018), which is particularly suited to our purpose here, since we will evaluate our semantic representations in the context of modeling…”
MethodBoosting Variational Generative Model via Condition Enhancing and Lexical-Editing
Tao, Z., Si, W., Li, J., Zhao, D., & Yan, R. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29908-8_30
Gao, S., Chen, X., Li, P., Chan, Z., Zhao, D., & Yan, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08837 · Cited by 4
“(Guu et al., 2018) proposed the first prototype editing model, which samples a prototype sentence from training data and then edits it into a new sentence.”
BackgroundLook-up and Adapt: A One-shot Semantic ParserPreprint
Lu, Z., Arabshahi, F., Labutov, I., & Mitchell, T. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12197 · Cited by 1
“(Guu et al., 2018) Gu et al. (2018).”
Static-Dynamically Attentive Variational Network for Dialogue Generation.
Chang, J., He, R., Wang, L., Wang, R., Zhao, X., & Yang, T. (2019)
Exemplar based Generation and Data Augmentation using Exemplar VAEsPreprint
Norouzi, S., Fleet, D. J., & Norouzi, M. (2020) · arXiv (Cornell University)
Deep Probabilistic Graphical ModelingPreprint
Dieng, A. B. (2020) · arXiv (Cornell University) · doi:10.7916/d8-gt4e-6m45 · Cited by 2
Predict and Use Latent Patterns for Short-Text ConversationPreprint
Chen, H.-T., Chao, Y.-C., Chao, T.-H., & Ma, W.-Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13982
“t al. 2018; Hossain, Ghazvininejad, and Zettlemoyer 2020), image caption generation (Kuznetsova et al. 2013; Mason and Charniak 2014), code generation (Hashimoto et al. 2018) and machine translation (Guu et al. 2018; Cao and Xiong 2018; Hossain, Ghazvininejad, and Zettlemoyer 2020). In dialogue generation specifically,…”
BackgroundSemi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding
Wang, G., & Tu, K. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.224
“Previous work has already broadly applied the autoencoder framework in many NLP tasks other than dependency parsing such as Part-Of-Speech (POS) tagging (Zhang et al., 2017a) and sentence generation (Guu et al., 2018).”
BackgroundGenerate and Revise: Reinforcement Learning in Neural Poetry
Zugarini, A., Pasqualini, L., Melacci, S., & Maggini, M. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533573
“Our generate-and-revise framework is related to retrieveand-edit seq2seq approaches [20]–[24], where text generation reduces to an adaptation/paraphrasing of the retrieved template(s) related to the current input.”
MethodWar of Words II: Enriched Models of Law-Making Processes
Kristof, V., Suresh, A., Grossglauser, M., & Thiran, P. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450131
“Edit modelling has been studied using neural models[12, 32] that suffer from the aforementioned issues of dataset size and interpretability.”
MethodTowards controllable neural generation of arguments
Hua, X. (2021) · Digital Repository Service (Northeastern University) · doi:10.17760/d20412319
Exploiting Explicit and Inferred Implicit Personas for Multi-turn Dialogue Generation
Wang, R., He, R., Wang, L., Si, Y., Liu, H., Wang, H., & Dang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88480-2_39
NeurInt : Learning to Interpolate through Neural ODEsPreprint
Bose, A., Das, A., Dandi, Y., & Rai, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04123
“…and target images in our model can also be interpreted as a form of external memory, which has been shown to improve generation quality in several works (Born-schein et al. 2017; Guu et al. 2018; Hoffman and Johnson 2016; Tomczak and Welling 2018b; Khandelwal et al. 2019; Li, Zhu, and Zhang 2016).”
Controllable Semantic Parsing via Retrieval AugmentationPreprint
Pasupat, P., Zhang, Y., & Guu, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08458
“Some methods, like CASPER, encode the retrievals alongside the query and let the model decides how to use them (Guu et al., 2018; Hashimoto et al., 2018; He et al., 2020; Weston et al., 2018; Pandey et al., 2018; Lewis et al., 2020).”
BackgroundMethodResultSelf-citationFew-Shot Named Entity Recognition via Label-Attention Mechanism
Pan, Y., Yang, J., Ming, H., Jiang, L., & An, N. (2023) · International Conference on Computing and Artificial Intelligence (ICCAI) · doi:10.1145/3594315.3594358
“They transform the sequence labeling problem into other problems to activate prior knowledge in pre-trained models [2, 11, 13, 29].”
BackgroundThe Fourth Workshop on Insights from Negative Results in NLP
Tafreshi, S., Akula, A., Sedoc, J., Валентинович, Д. А., Rogers, A., & Rumshisky, A. (2023) · doi:10.18653/v1/2023.insights-1 · Cited by 1
Edit Aware Representation Learning via Levenshtein Prediction
Marrese-Taylor, E., Reid, M., & Solano, A. (2023) · Workshop on Insights from Negative Results in NLP (Insights) · doi:10.18653/v1/2023.insights-1.6
“One such example is the work of Guu et al. (2018), who proposed a sentence-level generative model that first samples a prototype sentence and then edits it into a new sentence.”
BackgroundMethodLexiGuard: Elevating NLP robustness through effortless adversarial fortification
Omar, M. (2023) · Advances in Engineering Innovation · doi:10.54254/2977-3903/2/2023012
Dialogue Generation with Skeleton Extraction and Causal Intervention
Wan, Z. (2023) · International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · doi:10.1109/prai59366.2023.10332043
“For example, [13] proposes a text generation model based on secondary editing.”
BackgroundA Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generation
Forristal, J., Mireshghallah, F., Durrett, G., & Berg-Kirkpatrick, T. (2023) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-1.26 · Cited by 13
“Guu et al. (2018) uses editing of random sentences sampled from a corpus in place of autoregressive LMs to generate fluent natural language text.”
MethodMultilingual k-Nearest-Neighbor Machine Translation-Nearest-Neighbor Machine Translation
Stap, D., & Monz, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.571 · Cited by 6
Modeling Cross-Modal Semantic Transformations From Coarse to Fine in CLIP
Peng, Z., Qi, Z., Cao, Y. H., Kang, Y., & Lv, W. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3613997
“Notably, vMF distribution-based VAEs have demonstrated e ff ectiveness in sentence generation tasks [30], revealing the suitability of hyperspherical latent spaces for modeling directional data.”
Long-Tail Crisis in Nearest Neighbor Language Models
Nishida, Y., Morishita, M., Deguchi, H., Kamigaito, H., & Watanabe, T. (2025) · Journal of Natural Language Processing · doi:10.5715/jnlp.32.1272 · Cited by 1
ISA: Test Case Generation Based on Improved Simulated Annealing Algorithm
Ji, S., Shi, Y., Deng, J., & Zhang, P. (2025) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec66846.2025.00014
“…centered on Deep Neural Networks (DNNs) has become a research hotspot in the field of artificial intelligence, and has shown great potential for development and wide application value in many fields such as computer vision [1], natural language processing (NLP) [2] and speech recognition [3].”
Guo, D., Luo, Z., Bouguila, N., & Fan, W. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.108792
Hyperspherical Representation Learning of Axial Data via Axial VAEs with Watson Distribution
Luo, Z., Fan, W., Amayri, M., & Bouguila, N. (2026) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3805030
“Unlike Euclidean data, directional data are usually normalized to have unit norm (i.e., 𝐿 2 -normalized) and are represented as points on a hypersphere, which differs from the standard representation in Euclidean space [12, 15].”
Response selection for end-to-end retrieval-based dialogue systems
Boussaha, B. E. A. (2019) · theses.fr (ABES) · doi:10.70675/81045dfazc0f4z4eb2zaa8ezc9afd8f02bba
Wang, Z., Zhang, G., Shi, N., & Lin, C. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_3
Text Generation with and without Retrieval
Fan, H. (2021) · theses.fr (ABES) · doi:10.70675/87ca97d1z017cz478azac26z38a64f0a75e0
Context-Aware Slot Refinement with Diffusion Language Models
Wang, X. W., Wu, Q., Yang, S., Chen, J., & Li, Y. (2026) · World Conference on Computing and Communication Technologies (WCCCT) · doi:10.1109/wccct69960.2026.11549664
“Alter-natively, using large generative models to rewrite the entire utterance may fix some corrupted values [11], but full-sentence rewriting introduces unnecessary degrees of freedom and can cause semantic drift, paraphrasing artifacts, or unintended edits to already-correct non-slot content [12].”
REALM: Retrieval-Augmented Language Model Pre-Training
Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M.-W. (2020) · arXiv · doi:10.48550/arxiv.2002.08909 · Cited by 574
“Retrieve-and-edit with learned retrieval In order to better explain the variance in the input text and enable controllable generation, Guu et al. (2018) proposed a language model with the retrieve-and-edit framework (Hashimoto et al., 2018) that conditions on text with high lexical overlap.”
MethodSelf-citation
Metadata
- Authors
- Guu, K., Hashimoto, T. B., Oren, Y., & Liang, P.
- Affiliations
- Stanford University (US)
- Corresponding author
- Guu, Kelvin
- Hashimoto, Tatsunori B.
- Oren, Yonatan
- Liang, Percy
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2018-12
- Volume
- 6
- Pages
- 437-450
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2963018920
- Cited by
- 300
- References
- 29
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 7 · 2023: 30 · 2022: 30 · 2021: 48 · 2020: 56 · 2019: 72 · 2018: 44
- FWCI
- 30.55
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency
- Army Research Office (W911NF-15-1-, W911NF, W911NF-15)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- sentence generation, generative language models, sentence similarity, perplexity, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.