Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data
Open accessAbstract
Fine-tuned pre-trained language models can suffer from severe miscalibration for both in-distribution and out-of-distribution (OOD) data due to over-parameterization.To mitigate this issue, we propose a regularized fine-tuning method.Our method introduces two types of regularization for better calibration: (1) On-manifold regularization, which generates pseudo on-manifold samples through interpolation within the data manifold.Augmented training with these pseudo samples imposes a smoothness regularization to improve in-distribution calibration.(2) Off-manifold regularization, which encourages the model to output uniform distributions for pseudo off-manifold samples to address the over-confidence issue for OOD data.Our experiments demonstrate that the proposed method outperforms existing calibration methods for text classification in terms of expectation calibration error, misclassification detection, and OOD detection on six datasets.Our code can be found at https://github.com/Lingkai-Kong/ Calibrated-BERT-Fine-Tuning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fine-tuning a model can result in significant miscalibrations owing to over-parameterization.
“In a similar vein, it has been observed that fine-tuning a model can result in significant miscalibrations, owing to over-parameterization [61].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [61]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354 · see citing works
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.01626 · Cited by 7,563 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Miyato, T., Maeda, S., Koyama, M., & Ishii, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858821 · Cited by 2,860 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968 · see citing works
Predicting good probabilities with supervised learning
Niculescu-Mizil, A., & Caruana, R. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102430 · Cited by 1,884 · see citing works
(2018) · International Conference on Learning Representations · Cited by 1,621 · see citing works
Fast is better than free: Revisiting adversarial trainingPreprint
Wong, E., Rice, L., & Kolter, J. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2001.03994 · Cited by 1,444 · see citing works
Weight Uncertainty in Neural Network
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,280 · see citing works
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051 · see citing works
Obtaining well calibrated probabilities using Bayesian binning
Pakdaman Naeini, M., Cooper, G., & Hauskrecht, M. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9602 · Cited by 991 · see citing works
Mixup: Beyond empirical risk minimizationPreprint
Zhang, H. (2024) · arXiv (Cornell University) · doi:10.57702/dcy1c3gw · Cited by 923 · see citing works
Manifold Mixup: Better Representations by Interpolating Hidden StatesPreprint
Verma, V., Lamb, A., Beckham, C., Najafi, A., Mitliagkas, I., Courville, A. C., López-Paz, D., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05236 · Cited by 919 · see citing works
Deep Bayesian Active Learning with Image Data
Gal, Y., Islam, R., & Ghahramani, Z. (2017) · arXiv · doi:10.48550/arxiv.1703.02910 · Cited by 900 · see citing works
When does label smoothing help?
Müller, R., Kornblith, S., & Hinton, G. (2019) · arXiv · doi:10.48550/arxiv.1906.02629 · Cited by 859 · see citing works
Regularizing Neural Networks by Penalizing Confident Output Distributions
Pereyra, G., Tucker, G., Chorowski, J., Kaiser, Ł., & Hinton, G. (2017) · arXiv · doi:10.48550/arxiv.1701.06548 · Cited by 655 · see citing works
HDLTex: Hierarchical Deep Learning for Text Classification
Kowsari, K., Brown, D. E., Heidarysafa, M., Meimandi, K. J., Gerber, M. S., & Barnes, L. E. (2017) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2017.0-134 · Cited by 484 · see citing works
On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks
Thulasidasan, S., (NNSA), U. N. N. S. A., Chennupati, G., Bilmes, J., Tanmoy Bhattacharya, A., & Michalak, S. (2019) · Neural Information Processing Systems · doi:10.2172/1525811 · Cited by 423 · see citing works
Deep Bayesian Active Learning with Image Data
Ghahramani, Z., Gal, Y., & Islam, R. (2017) · PMLR · doi:10.17863/cam.11070 · Cited by 392 · see citing works
Deep Active Learning for Named Entity Recognition
Shen, Y., Yun, H., Lipton, Z. C., Kronrod, Y., & Anandkumar, A. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2630 · Cited by 389 · see citing works
Multiplicative Normalizing Flows for Variational Bayesian Neural NetworksPreprint
Louizos, C., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01961 · Cited by 254 · see citing works
Siddhant, A., & Lipton, Z. C. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1318 · Cited by 239 · see citing works
Disentangling Adversarial Robustness and Generalization
Stutz, D., Hein, M., & Schiele, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00714 · Cited by 223 · see citing works
Importance of semantic representation: dataless classification
Chang, M., Ratinov, L., Roth, D., & Srikumar, V. (2008) · National Conference on Artificial Intelligence · Cited by 222 · see citing works
Trainable Calibration Measures for Neural Networks from Kernel Mean Embeddings
Kumar, A., Sarawagi, S., & Jain, U. (2018) · International Conference on Machine Learning · Cited by 137 · see citing works
Deep Learning for NLP (without Magic)
Socher, R., Bengio, Y., & Manning, C. D. (2012) · Cited by 131 · see citing works
Learning to Defend by Learning to AttackPreprint
Jiang, H., Chen, Z., Shi, Y., Dai, B., & Zhao, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01213 · Cited by 60 · see citing works
Self-citationDeep Active Learning for Named Entity Recognition.
Shen, Y., Yun, H., Lipton, Z. C., et al. (2018) · CaltechAUTHORS (California Institute of Technology) · Cited by 46 · see citing works
Learning to Defense by Learning to Attack.
Chen, Z., Jiang, H., Shi, Y., et al. (2019) · International Conference on Learning Representations · Cited by 6 · see citing works
Self-citationA Deep Network with Visual Text Composition Behavior
Guo, H. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-2059 · Cited by 4 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 6
- Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S Schoenholz, Maithra Raghu, Martin Wattenberg, Ian Goodfellow, and G Brain. 2018. The relationship between high-dimensional geometry and adversarial examples. arXiv preprint arXiv:1801.02774. · arXiv
- Haoming Jiang, Pengcheng He, Weizhu Chen, Xi- aodong Liu, Jianfeng Gao, and Tuo Zhao. 2020. SMART: Robust and efficient fine-tuning for pre- trained natural language models through principled regularized optimization. In Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, pages 2177-2190.
- Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predic- tive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pages 6402-6413.
- Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems, pages 7167-7177.
- Christos Louizos and Max Welling. 2017. Multiplica- tive normalizing flows for variational Bayesian neu- ral networks. In International Conference on Ma- chine Learning, pages 2218-2227.
- Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanx- ing Zhu, and Bin Dong. 2019. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Pro- cessing Systems, pages 227-238.
Cited by
Sorted by citations
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644 · see citing works
Self-citationJiang, Z., Araki, J., Ding, H., & Neubig, G. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00407 · Cited by 177 · see citing works
A Survey of Confidence Estimation and Calibration in Large Language Models . -
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.366 · Cited by 88 · see citing works
A Close Look into the Calibration of Pre-trained Language Models
Chen, Y., Yuan, L., Cui, G., Liu, Z., & Ji, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.75 · Cited by 22 · see citing works
- Authors
- Kong, L., Jiang, H., Zhuang, Y., Lyu, J., Zhao, T., & Zhang, C.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 1326-1340
- OpenAlex
- W3099142828
- Cited by
- 58
- References
- 41
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 7 · 2023: 18 · 2022: 13 · 2021: 12 · 2020: 1
- FWCI
- 4.14
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2008334)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- out-of-distribution detection, pseudo-samples, interpolation, overconfidence, text classification, misclassification detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
