An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Text classification approaches have usually required task-specific model architectures and huge labeled datasets.Recently, thanks to the rise of text-based transfer learning techniques, it is possible to pre-train a language model in an unsupervised manner and leverage them to perform effectively on downstream tasks.In this work we focus on Japanese and show the potential use of transfer learning techniques in text classification.Specifically, we perform binary and multi-class sentiment classification on the Rakuten product review and Yahoo movie review datasets.We show that transfer learning-based approaches perform better than task-specific models trained on 3 times as much data.Furthermore, these approaches perform just as well for language modeling pre-trained on 1 30 of Wikipedia.We release our pre-trained models and code as open source.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“And they show a noticeable improvement in supervised, semi-supervised, and unsupervised MT.”
From Pre-trained models for natural language processing: A survey (2020) · cites this paper as [192]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“, 2018), ULMFiT (Howard and Ruder, 2018), and BERT (Devlin et al., 2018) to each pretrain a language model which can then be used to perform downstream tasks.”
MethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“The basic idea is similar to how fine-tuning ImageNet (Deng et al., 2009) helps many computer vision tasks (Huh et al.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“, 2018), ULMFiT (Howard and Ruder, 2018), and BERT (Devlin et al., 2018) to each pretrain a language model which can then be used to perform downstream tasks.”
MethodGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 2×
“There have been many neural network implementations, including word2vec (Mikolov et al., 2013), fasttext (Joulin et al., 2016) and Glove (Pennington et al., 2014) that embed using a single layer and achieve state-of-the-art performance in various NLP tasks.”
BackgroundEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“There have been many neural network implementations, including word2vec (Mikolov et al., 2013), fasttext (Joulin et al.”
BackgroundConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 3×
“In the past few years, researchers have made significant progress from models that make use of deep learning techniques.(Kim, 2014; Lai et al., 2015; Chen et al., 2017; Lin et al., 2017).”
BackgroundMethodDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 4×
“To see if existing research can make progress in Japanese, we make use of recent transfer learning models such as ELMo (Peters et al., 2018), ULMFiT (Howard and Ruder, 2018), and BERT (Devlin et al.”
BackgroundMethodInfluentialRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · cited 2×
“(2017) with ELMo as it is known to be state-of-the-art11 on SST (Socher et al., 2013) datasets.”
BackgroundMethodSelf-citationBag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
Research works printed but not linked: 2
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proc. of NAACL.
- Maria Pontiki, Dimitris Galanis, Haris Papageor- giou, Ion Androutsopoulos, Suresh Manandhar, AL- Smadi Mohammad, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, et al. 2016. Semeval-2016 task 5: Aspect based sentiment anal- ysis. In Proceedings of the 10th international work- shop on semantic evaluation (SemEval-2016), pages 19-30.
Cited by
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591
“Bataa and Wu [9] utilized BERT with transfer learning techniques and achieve new state-of-the-art in Japanese SA.”
MethodA Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification
Qasim, R., Bangyal, W. H., Alqarni, M. A., & Ali Almazroi, A. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/3498123 · Cited by 236
“Researchers performed an evaluation of sentiment analysis approaches based on transfer learning for the Japanese dataset [30].”
MethodFoundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Alhawawreh, M., Berdik, D., & Jararweh, Y. (2024) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 185
“[16] utilised BERT with transfer learning-based approaches and achieved an outstanding result in Japanese SA.”
State of the art: a review of sentiment analysis based on sequential transfer learning
Chan, J. Y.-L., Bea, K. T., Leow, S. M. H., Phoong, S. W., & Cheng, W. K. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10183-8 · Cited by 179
“Likewise, fine-tuned BERT has also achieved great results in Japanese sentiment analysis (Bataa and Wu 2019).”
ResultLossio-Ventura, J. A., Weger, R., Lee, A. Y., Guinee, E. P., Chung, J., Atlas, L., Linos, E., & Pereira, F. (2024) · JMIR Mental Health · doi:10.2196/50150 · Cited by 73
Metadata
- Authors
- Bataa, E., & Wu, J.
- Affiliations
- Exawizards Inc. (Japan) (JP)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 4652-4657
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2944958965
- Cited by
- 35
- References
- 23
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 5 · 2023: 9 · 2022: 6 · 2021: 7 · 2020: 5
- FWCI
- 1.72
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- transfer learning, sentiment analysis, Japanese language, text classification, pre-trained language models, unsupervised pre-training, Wikipedia, product reviews, movie reviews, multi-class sentiment classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese. https://app.cochat.ai/papers/10.18653/v1/p19-1458
- The paper
- Bataa, E., & Wu, J. (2019). An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/p19-1458
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
