Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
Large multi-label datasets contain labels that occur thousands of times (frequent group), those that occur only a few times (few-shot group), and labels that never appear in the training dataset (zero-shot group). Multi-label few- and zero-shot label prediction is mostly unexplored on datasets with large label spaces, especially for text classification. In this paper, we perform a fine-grained evaluation to understand how state-of-the-art methods perform on infrequent labels. Furthermore, we develop few- and zero-shot methods for multi-label text classification when there is a known structure over the label space, and evaluate them on two publicly available medical text datasets: MIMIC II and MIMIC III. For few-shot labels we achieve improvements of 6.2% and 4.8% in R@10 for MIMIC II and MIMIC III, respectively, over prior efforts; the corresponding R@10 improvements for zero-shot labels are 17.3% and 19%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“For training we use the ADAM (Kingma and Ba, 2015) optimizer with a minibatch size of 8 and a learning rate of 0.001. q , the GCNN hidden layer size, is set to 300.”
MethodInfluentialSemi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 3×
“Neural networks (NNs) perform well for many small-scale classification tasks (Kim, 2014; Kalchbrenner et al., 2014).”
BackgroundMethod(2010) · International Conference on Machine Learning · Cited by 13,995
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · cited 2×
“In Figure 1, we plot the label frequency distribution of diagnosis and procedure labels for the entire MIMIC III (Johnson et al., 2016) dataset.”
MethodModeling Relational Data with Graph Convolutional Networks
Schlichtkrull, M., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_38 · Cited by 5,570
Siamese Neural Networks for One-shot Image Recognition
Koch, G. E., Zemel, R. S., & Salakhutdinov, R. (2015) · Cited by 5,270
Prototypical Networks for Few-shot Learning
Snell, J., Swersky, K., & Zemel, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05175 · Cited by 4,755 · cited 2×
“Prototypical Networks (Snell et al., 2017) average all instances in each class to form “prototype label vectors” and train using a traditional cross-entropy loss.”
BackgroundHierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712 · cited 4×
“Finally, Mullenbach et al. (2018) expand on prior ACNNs (Yang et al., 2016; Allamanis et al., 2016) to develop a labelwise attention framework where the most informative ngrams are extracted for each label in the dataset.”
BackgroundInfluentialAdvances In Deep Learning On Graphs (Gsp'18 Workshop)
Defferrard, M. (2016) · Figshare · doi:10.5281/zenodo.1286817 · Cited by 4,613
Cited by
Generalizing from a Few Examples: A Survey on Few-Shot Learning
Wang, Y., Yao, Q., Kwok, J. T.-Y., & Ni, L. M.-S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05046 · Cited by 772
Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
Yin, W., Hay, J., & Roth, D. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1404 · Cited by 537
“For example, Rios and Kavuluru (2018) work on medical data while reporting R@K as metric; Xia et al. (2018) work on SNIPS-NLU intent detection data while only unseen intents are in the label-searching space in evaluation.”
BackgroundThe Emerging Trends of Multi-Label Learning
Liu, W., Wang, H., Shen, X., & Tsang, I. W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119334 · Cited by 400
Text Classification Using Label Names Only: A Language Model Self-Training Approach
Meng, Y., Zhang, Y., Huang, J., Xiong, C., Ji, H., Zhang, C., & Han, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.724 · Cited by 196
A survey on machine learning from few samples
Lu, J., Gong, P., Ye, J., Zhang, J., & Zhang, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109480 · Cited by 100
Revisiting Transformer-based Models for Long Document Classification
Dai, X., Chalkidis, I., Darkner, S., & Elliott, D. (2022) · arXiv · doi:10.48550/arxiv.2204.06683 · Cited by 58
“, 2020), code’s textual descriptions (Mullenbach et al., 2018; Rios and Kavuluru, 2018).”
Liu, J., Capurro, D., Nguyen, A., & Verspoor, K. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00474-9 · Cited by 50
Kaur, R., Ginige, J. A., & Obst, O. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118997 · Cited by 48
Ahmed, T., Ivan, S., Kabir, M., Mahmud, H., & Hasan, K. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00934-4 · Cited by 42
“In recent studies, few-shot learning is being used in the domain of text classification where the class labels are imbalanced or infrequent and proved to be effective Rios and Kavuluru (2018).”
BackgroundInfluentialRetrieval-Based Diagnostic Decision Support: Mixed Methods Study
Abdullahi, T., Mercurio, L., Singh, R., & Eickhoff, C. (2024) · JMIR Medical Informatics · doi:10.2196/50209 · Cited by 20
“However, most clinical cohorts exhibit imbalanced class distributions, characterized by a long-tailed pattern [15,16] in which certain diagnostic classes represent most training samples whereas others exhibit few or even 0 data points.”
Background
Metadata
- Authors
- Rios, A., & Kavuluru, R.
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Volume
- 2018
- Pages
- 3132-3142
- OpenAlex
- W2889577585
- Cited by
- 181
- References
- 46
- FWCI
- 8.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.