Cited by 129Topic ModelingSentiment Analysis and Opinion MiningMultimodal Machine Learning Applications

The DOI matches its Crossref record; 49 of the 54 references it cites were found in a registry or an index.

Generating Datasets with Pretrained Language Models

Open access

Schick, Timo, Schütze, Hinrich·2021·Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

DOI 10.18653/v1/2021.emnlp-main.555

Open this paper in CoChat — free account Open PDF

Abstract

To obtain high-quality sentence embeddings from pretrained language models (PLMs), they must either be augmented with additional pretraining objectives or finetuned on a large set of labeled text pairs.While the latter approach typically outperforms the former, it requires great human effort to generate suitable datasets of sufficient size.In this paper, we show how PLMs can be leveraged to obtain high-quality sentence embeddings without the need for labeled data, finetuning or modifications to the pretraining objective: We utilize the generative abilities of large and high-performing PLMs to generate entire datasets of labeled text pairs from scratch, which we then use for finetuning much smaller and more efficient models.Our fully unsupervised approach outperforms strong baselines on several semantic textual similarity datasets.1

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.18653/v1/2021.emnlp-main.555

Cites

49 of 54 references verified0 citation errors · 5 not checkable

Statistics

Table 3: Comparison of similarity scores in STS--x 2 to human judgments for 100 examples. Examples are chosen randomly from the version of STS--x 2 used for training (including label smoothing, augmentation with random pairs and removal of examples where x 1 = x 2 ). For column i and row j , the value shown is the percentage of examples generated by DINO for similarity score i that were assigned score j in our human evaluation.CC-BY · From the PDF

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Untitled work
(2017) · Cited by 11,625
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645

Show all 47 references

Research works printed but not linked: 1

Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.

Sorted by citations

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
“In this way, by selecting the appropriate prompts we can manipulate the model behavior so that the pre-trained LM itself can be used to predict the desired output, sometimes even without any additional task-specific training (Table 1(d); e.g., Brown et al. [9], Petroni et al. [100], Radford et al. [105], Schick and Sc…”
MethodInfluential
Text Data Augmentation for Deep Learning
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00492-0 · Cited by 1,852
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150
Self-citation
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
“In this way, by selecting the appropriate prompts we can manipulate the model behavior so that the pre-trained LM itself can be used to predict the desired output, sometimes even without any additional task-specific training (Table 1(d); e.g., Brown et al. [9], Petroni et al. [100], Radford et al. [105], Schick and Sc…”
MethodInfluential
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
West, P., Bhagavatula, C., Hessel, J., Hwang, J. D., Jiang, L., Bras, R. J. L., Lu, X., Welleck, S., & Choi, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.341 · Cited by 209
Automated data processing and feature engineering for deep learning and big data applications: A survey
Mumuni, A., & Mumuni, F. (2024) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2024.01.002 · Cited by 221
A Holistic Approach to Undesired Content Detection in the Real World
Markov, T., Zhang, C., Agarwal, S., Nekoul, F. E., Lee, T. T. H., Adler, S., Jiang, A., & Weng, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26752 · Cited by 264
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
Liu, A., Swayamdipta, S., Smith, N. A., & Choi, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.508 · Cited by 178
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor
Honovich, O., Scialom, T., Levy, O., & Schick, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.806 · Cited by 324
Self-citation
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
Ye, J., Gao, J., Li, Q., Xu, H., Feng, J., Wu, Z., Yu, C., & Kong, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.801 · Cited by 200
Revisiting DocRED - Addressing the False Negative Problem in Relation Extraction
Tan, Q., Xu, L., Bing, L., Ng, H. T., & Aljunied, S. M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.580 · Cited by 94
“Another line of work utilizes manually designed prompts and instructions to generate data in an un-supervised manner (Wang et al., 2019; Schick and Schütze, 2021; Chia et al., 2022).”
FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering
Li, Z., Fan, S., Gu, Y., Li, X., Duan, Z., Dong, B., Liu, N., & Wang, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29823 · Cited by 121
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers
Kim, B., Kim, H., Lee, S., Lee, G., Kwak, D., Hyeon, J. D., Park, S., Kim, S., Kim, S., Seo, D., et al. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.274 · Cited by 120
True Few-Shot Learning with Prompts—A Real-World Perspective
Schick, T., & Schütze, H. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00485 · Cited by 81
Self-citation
DetGPT: Detect What You Need via Reasoning
Pi, R., Gao, J., Diao, S., Pan, R., Dong, H., Zhang, J., Yao, L., Han, J., Xu, H., Kong, L., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.876 · Cited by 127
LLMaAA: Making Large Language Models as Active Annotators
Zhang, R., Li, Y., Ma, Y., Zhou, M. Q., & Zou, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.872 · Cited by 114
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53
“Our work pushes the above direction further, by examining whether generated data serves as high-quality evaluation data; useful generated training data can contain noisy or mislabeled examples (Schick and Schütze, 2021), while useful evaluation data must contain relatively few errors.”
BackgroundMethod
AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation
Zheng, C., Sabour, S., Wen, J., Zhang, Z., & Huang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.99 · Cited by 84
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets
Wu, Y., Gardner, M., Stenetorp, P., & Dasigi, P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.190 · Cited by 69
Annotated dataset creation through large language models for non-english medical NLP
Frei, J., & Krämer, F. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104478 · Cited by 56
Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step
Li, L. H., Hessel, J., Yu, Y., Ren, X., Chang, K.-W., & Choi, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.150 · Cited by 148
Distributed Inference and Fine-tuning of Large Language Models Over The Internet
Borzunov, A., Ryabinin, M., Chumachenko, A., Baranchuk, D., Dettmers, T., Belkada, Y., Samygin, P., & Raffel, C. (2023) · arXiv · doi:10.48550/arxiv.2312.08361 · Cited by 35
Language Models in the Loop: Incorporating Prompting into Weak Supervision
Smith, R., Fries, J., Hancock, B. J., & Bach, S. H. (2024) · ACM / IMS Journal of Data Science · doi:10.1145/3617130 · Cited by 68
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models
Vulić, I., Su, P.-H., Coope, S., Gerz, D., Budzianowski, P., Casanueva, I., Mrkšić, N., & Wen, T.-H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.88 · Cited by 34
GPS: Genetic Prompt Search for Efficient Few-Shot Learning
Xu, H., Chen, Y., Du, Y., Shao, N., Yanggang, W., Li, H., & Yang, Z. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.559 · Cited by 69
Fine-Tuned Large Language Model for Visualization System: A Study on Self-Regulated Learning in Education
Gao, L., Lu, J., Shao, Z., Lin, Z., Yue, S., Ieong, C., Sun, Y., Zauner, R. J., Wei, Z., & Chen, S. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456145 · Cited by 44
ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing
Liu, R., & Shah, N. B. (2023) · arXiv · doi:10.48550/arxiv.2306.00622 · Cited by 23
“Data generation has seen advances in either simplifying or fully automating the creation of questions and labels [Par+22; Liu+22; SS21; Wan+21].”
Method
The Role of Federated Learning in a Wireless World with Foundation Models
Chen, Z., Yang, H. H., Tay, Y. C., Chong, K. F. E., & Quek, T. Q. S. (2024) · IEEE Wireless Communications · doi:10.1109/mwc.005.2300481 · Cited by 33
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Ye, J., Gao, J., Wu, Z., Feng, J., Yu, C., & Kong, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.269 · Cited by 48
Generating Better Items for Cognitive Assessments Using Large Language Models
Laverghetta, A., & Licato, J. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.34 · Cited by 27
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
Vu, T., Luong, M.-T., Le, Q. V., Simon, G., & Iyyer, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.462 · Cited by 45
Improving Contrastive Learning of Sentence Embeddings from AI Feedback
Cheng, Q., Yang, X., Sun, T., Li, L., & Qiu, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.707 · Cited by 36
Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models
Zhang, Q., Naradowsky, J., & Miyao, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.417 · Cited by 36
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval
Yu, Y., Zhuang, Y., Zhang, R., Yu, M., Shen, J., & Zhang, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.748 · Cited by 28
Explainable AI: XAI-guided context-aware data augmentation
Mersha, M. A., Yigezu, M. G., Tonja, A. L., Shakil, H., Iskandar, S., Kolesnikova, O., & Kalita, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128364 · Cited by 24
STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models
Ma, M. D., Wang, X., Kung, P.-N., Brantingham, P. J., Peng, N., & Wang, W. W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29839 · Cited by 22
Differentially Private Language Models for Secure Data Sharing
Mattern, J., Jin, Z., Weggenmann, B., Schölkopf, B., & Sachan, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.323 · Cited by 49
UniHD at TSAR-2022 Shared Task: Is Compute All We Need for Lexical Simplification?
Aumiller, D., & Gertz, M. (2022) · Workshop on Text Simplification, Accessibility and Readability (TSAR) · doi:10.18653/v1/2022.tsar-1.28 · Cited by 20
Synthetically generated text for supervised text analysis
Halterman, A. (2025) · Political Analysis · doi:10.1017/pan.2024.31 · Cited by 22
A Robust Bias Mitigation Procedure Based on the Stereotype Content Model
Ungless, E. L., Rafferty, A., Nag, H., & Roß, B. (2022) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2022.nlpcss-1.23 · Cited by 16
Text Data Augmentation for the Korean Language
Vu, D. T., Yu, G.-H., Lee, C.-W., & Kim, J. Y. (2022) · Applied Sciences · doi:10.3390/app12073425 · Cited by 13
Don’t Prompt, Search! Mining-based Zero-Shot Learning with Language Models
van de Kar, M., Xia, M., Chen, D., & Artetxe, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.509 · Cited by 13
Contrastive Learning of Sentence Embeddings from Scratch
Zhang, J., Lan, Z., & He, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.238 · Cited by 26
Geographic and Geopolitical Biases of Language Models
Faisal, F., & Anastasopoulos, A. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.12 · Cited by 27
Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Framework
Chen, Y., Zhang, Y., Wang, B., Liu, Z., & Li, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.558 · Cited by 13
Data Augmentation for Low-Resource Dialogue Summarization
Liu, Y., Maynez, J., Simões, G., & Narayan, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.53 · Cited by 13
ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset
Jin, Z., Wang, X., Cheng, F., Sun, C., Liu, Q., & Qu, H. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3236380 · Cited by 12
Team:PULSAR at ProbSum 2023:PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients’ Problems and Data Augmentation with Black-box Large Language Models
Li, H., Wu, Y., Schlegel, V., Batista-Navarro, R., Nguyen, T.-T., Kashyap, A. R., Zeng, X., Beck, D., WINKLER, S. S., & Nenadić, G. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.49 · Cited by 12
Hybrid Data Augmentation for Citation Function Classification
Zhang, Y., Wang, Y., Sheng, Q. Z., Mahmood, A., Zhang, W. E., & Zhao, R. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191695 · Cited by 11
Math Word Problem Solving by Generating Linguistic Variants of Problem Statements
Raiyan, S. R., Faiyaz, M. N., Kabir, S., Kabir, M., Mahmud, H., & Hasan, M. K. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.49 · Cited by 13
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models
Li, S., Li, L., Liu, Y., Ren, S., Liu, Y., Gao, R., Sun, X., & Hou, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72897-6_19 · Cited by 29
Counterfactual Data Augmentation via Perspective Transition for Open-Domain Dialogues
Ou, J., Zhang, J., Feng, Y., & Zhou, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.106 · Cited by 12
Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation
Sclar, M., West, P., Kumar, S., Tsvetkov, Y., & Choi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.655 · Cited by 14
Unsupervised Neural Machine Translation with Generative Language Models OnlyPreprint
Han, J. M., Babuschkin, I., Edwards, H. S., Neelakantan, A., Xu, T., Polu, S., Ray, A., Shyam, P., Ramesh, A., Radford, A., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05448 · Cited by 22
Liberating Seen Classes: Boosting Few-Shot and Zero-Shot Text Classification via Anchor Generation and Classification Reframing
Liu, H., Zhao, S., Zhang, X., Zhang, F., Wang, W. W., Ma, F., Chen, H., Hong, Y., & Zhang, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29827 · Cited by 13
Towards Generating High-Quality Knowledge Graphs by Leveraging Large Language Models
Ezzabady, M. K., Ieng, F., Khorashadizadeh, H., Benamara, F., Groppe, S., & Sahri, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70239-6_31 · Cited by 10
Relation Extraction in Underexplored Biomedical Domains: A Diversity-optimized Sampling and Synthetic Data Generation Approach
Delmas, M., Wysocka, M., & Freitas, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00520 · Cited by 13
Annotation-preserving machine translation of English corpora to validate Dutch clinical concept extraction tools
Seinen, T. M., Kors, J. A., van Mulligen, E. M., & Rijnbeek, P. R. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae159 · Cited by 8
Enhancing legal question answering with data generation and knowledge distillation from large language models
Italiani, P., Moro, G., & Ragazzi, L. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09463-9 · Cited by 15
Bringing legal knowledge to the public by constructing a legal question bank using large-scale pre-trained language model
Yuan, M., Kao, B., Wu, T.-H., Cheung, M. M. K., Chan, H. W. H., Cheung, A. S. Y., Chan, F. W. H., & Chen, Y. (2023) · Artificial Intelligence and Law · doi:10.1007/s10506-023-09367-6 · Cited by 8
Leveraging QA Datasets to Improve Generative Data Augmentation
Mekala, D., Vu, T., Schick, T., & Shang, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.660 · Cited by 10
Self-citation
Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation
Yang, K., Deng, O., Chen, C., Shin, R., Roy, S., & Durme, B. V. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.291 · Cited by 9
Utilizing Large Language Models to Synthesize Product Desirability Datasets
Hastings, J. D., Weitl-Harms, S., Doty, J., Myers, Z. J., & Thompson, W. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826001 · Cited by 7
Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networks
Lin, Y. T., Papangelis, A., Kim, S., & Hakkani-Tür, D. Z. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.3 · Cited by 7
Universal Self-Adaptive Prompting
Wan, X., Sun, R., Nakhost, H., Dai, H., Eisenschlos, J. M., Arık, S. Ö., & Pfister, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.461 · Cited by 21
From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery
Chen, Y., Xi, N., Du, Y., Wang, H., Chen, J., Zhao, S., & Qin, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30198 · Cited by 11
A Logical Fallacy-Informed Framework for Argument Generation
Mouchel, L., Paul, D., Cui, S., West, R., Bosselut, A., & Faltings, B. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.374 · Cited by 7
Does synthetic data augmentation improve the performances of machine learning classifiers for identifying health problems in patient–nurse verbal communications in home healthcare settings?
Scroggins, J. K., Topaz, M., Song, J., & Zolnoori, M. (2024) · Journal of Nursing Scholarship · doi:10.1111/jnu.13004 · Cited by 7
MotionGPT: Human Motion Synthesis with Improved Diversity and Realism via GPT-3 Prompting
Ribeiro-Gomes, J., Cai, T., Milacski, Z. Á., Wu, C. H., Prakash, A., Takagi, S., Aubel, A., Kim, D., Bernardino, A. J. M., & la Torre, F. D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00499 · Cited by 12
ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification
Guo, K., Yu, H., Liao, C., Li, J., & Zhang, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.200 · Cited by 6
What happens before and after: Multi-Event Commonsense in Event Coreference Resolution
Ravi, S., Tanner, C., Ng, R. T., & Shwartz, V. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.125 · Cited by 10
Data Augmentation with Large Language Models for Vietnamese Abstractive Text Summarization
Le, H. M., Luong, V. T., & Luong, N. H. (2023) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr59823.2023.10288906 · Cited by 5
Leveraging Large Language Models for Structure Learning in Prompted Weak Supervision
Su, J., Yu, P., Zhang, J., & Bach, S. H. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386190 · Cited by 5
Learning Evaluation Models From Large Language Models for Sequence Generation
Wang, C., Zhou, H., Chang, K., Liu, T., Zhang, C., Yang, M., Du, Q., Xiao, T., Zhang, Y., & Zhu, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3587460 · Cited by 11
Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisation
Li, H., Schlegel, V., Batista-Navarro, R., & Nenadić, G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.786 · Cited by 13
Alfred: A System for Prompted Weak Supervision
Yu, P., & Bach, S. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.46 · Cited by 7
Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task
Lee, N., Park, C., Choi, H.-J., & Choo, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.733 · Cited by 4
Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation
Kim, B., Seo, S., Han, S., Erdenee, E., & Chang, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.286 · Cited by 6
An Empirical Analysis of Leveraging Knowledge for Low-Resource Task-Oriented Semantic Parsing
Kulkarni, M., Zhong, A., Mesnards, N. G. D., Movaghati, S., Sridhar, M., Xie, H., & Lu, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.123 · Cited by 3
Reliable Data Generation and Selection for Low-Resource Relation Extraction
Yu, J., Wang, X., & Chen, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29915 · Cited by 4
Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model
Li, M., Zhang, R., Nie, Z., & Mao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29263 · Cited by 3
AuToGen: Automated Tool Learning Data Generation with Domain-specific Structured Data
Zeng, D., Zhou, L., Zhang, Z., & Jiang, L. (2024) · Data Intelligence · doi:10.3724/2096-7004.di.2024.0005 · Cited by 3
WikiOmnia: filtration and evaluation of the generated QA corpus on the whole Russian Wikipedia
Pisarevskaya, D., & Shavrina, T. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.10 · Cited by 3
A Quantum Language-Inspired Tree Structural Text Representation for Semantic Analysis
Yan, Y., Qiu, D., & Yan, R. (2022) · Mathematics · doi:10.3390/math10060914 · Cited by 2
Collective Human Opinions in Semantic Textual Similarity
Wang, Y., Tao, S., Xie, N., Yang, H., Baldwin, T., & Verspoor, K. M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00584 · Cited by 7
Distance Matters: Euclidean Embedding Distances for Improved Language Model Generalization and Adaptability
Alshamrani, S. (2024) · IEEE Access · doi:10.1109/access.2024.3434612 · Cited by 2
SumREN: Summarizing Reported Speech about Events in News
Reddy, R. G., Elfardy, H., Chan, H. P., Small, K., & Ji, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26506 · Cited by 3
BARLE: Background-Aware Representation Learning for Background Shift Out-of-Distribution Detection
Duan, H., Yang, Y., Abbasi, A., & Tam, K. Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.53 · Cited by 2
Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMs
Golde, J., Haller, P., Hamborg, F., Risch, J., & Akbik, A. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.1 · Cited by 2
Data Augmentation with GPT-3.5 for Vietnamese Natural Language Inference
Mai, H.-H., & Luong, N. H. (2023) · RIVF International Conference on Computing and Communication Technologies (RIVF) · doi:10.1109/rivf60135.2023.10471880 · Cited by 1
Unveiling the Evolution and Impact of Pretrained Language Models in Natural Language Processing
Bian, H. (2024) · International Conference on Computer, Artificial Intelligence and Control Engineering (CAICE) · doi:10.1145/3672758.3672854 · Cited by 1
Towards enhancing Natural Language Inference for identifying erroneous outputs in data-to-text generation
Ok, C., Lee, E., Ju, H., Lee, J., & Oh, D. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115961 · Cited by 1
Large Models are Good Annotators for Zero-Shot Learning
He, Q., Jia, Y., Li, W., Liao, S., Quan, R., Cui, T., & Qin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730219 · Cited by 1
Predicting organized collective violence through hostile discourse in social media
Mendieta, M. V., Warren, T. C., & Vaca, C. (2025) · EPJ Data Science · doi:10.1140/epjds/s13688-025-00596-0 · Cited by 3
Assessing the quality and performance of synthetic data augmentation to identify stigmatizing language in obstetric clinical notes
Scroggins, J. K., Barcelona, V., Hulchafo, I. I., Harkins, S., Zhang, Z., Scharp, D., Cato, K., Moen, H., Davoudi, A., Tadiello, M., et al. (2026) · Nursing Outlook · doi:10.1016/j.outlook.2026.102757 · Cited by 1
The Versatility of Large Language Models: A Comprehensive Review and Structured Survey of Architectures, Applications, Challenges, and Future Trajectories
Peykani, P., Charles, V., Emrouznejad, A., Ghanidel, S., & Sabour, D. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10720-6 · Cited by 1
I-WAS: A Data Augmentation Method with GPT-2 for Simile Detection
Chang, Y., Zhang, R., & Pu, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41682-8_17 · Cited by 1
CHEF in the Language Kitchen: A Generative Data Augmentation Leveraging Korean Morpheme Ingredients
Seo, J., Moon, H., Lee, J., Eo, S., Park, C., & Lim, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.367 · Cited by 1

Authors: Schick, T., & Schütze, H.
Affiliations: Ludwig-Maximilians-Universität München (DE)
ORCID: Schütze, Hinrich
Venue: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Publisher: Association for Computational Linguistics
Published: 2021
Pages: 6943-6951
DOI: 10.18653/v1/2021.emnlp-main.555
OpenAlex: W3154863804
Cited by: 122
References: 47
Citations by year: 2026: 7 · 2025: 15 · 2024: 27 · 2023: 39 · 2022: 26 · 2021: 8
FWCI: 11.55
Type: conference paper
Language: en
Open access: hybrid
doi.org
Licence: cc-by
Locations: Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
arXiv (Cornell University) (submittedVersion)
Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, other-oa)
arXiv (Cornell University)
Universitätsbibliothek der LMU
Funders: H2020 European Research Council (740516)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: sentence embeddings, pre-trained language models, semantic textual similarity, generative language models, fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Generating Datasets with Pretrained Language Models

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks