The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Code Mixing: A Challenge for Language Identification in the Language of Social Media
Open accessAbstract
In social media communication, multilingual speakers often switch between languages, and, in such an environment, automatic language identification becomes both a necessary and challenging task.In this paper, we describe our work in progress on the problem of automatic language identification for the language of social media.We describe a new dataset that we are in the process of creating, which contains Facebook posts and comments that exhibit code mixing between Bengali, English and Hindi.We also present some preliminary word-level language identification experiments using this dataset.Different techniques are employed, including a simple unsupervised dictionary-based approach, supervised word-level classification with and without contextual clues, and sequence labelling using Conditional Random Fields.We find that the dictionary-based approach is surpassed by supervised classification and sequence labelling, and that it is important to take contextual clues into consideration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
A Practical Guide to Support Vector Classication
Hsu, C., Chang, C.-C., & Lin, C. (2008) · Cited by 5,046
Language identification in the limit
Gold, E. M. (1967) · Information and Control · doi:10.1016/s0019-9958(67)91165-5 · Cited by 3,572
N-gram-based text categorization
Cavnar, W. B., & Trenkle, J. M. (1994) · Cited by 1,497
Code Switching in Conversation: Language, Interaction and Identity
Lanvers, U., & Auer, P. (2000) · The Modern Language Review · doi:10.2307/3736717 · Cited by 1,293
SemEval-2016 Task 4: Sentiment Analysis in Twitter
Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., & Stoyanov, V. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1001 · Cited by 966
Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters
Owoputi, O., O'Connor, B., Dyer, C., Gimpel, K., Schneider, N., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473408 · Cited by 662
The BNC Handbook: Exploring the British National Corpus with SARA
Aston, G., & Burnard, L. (1998) · Cited by 652
SemEval-2014 Task 9: Sentiment Analysis in Twitter
Rosenthal, S., Ritter, A., Nakov, P., & Stoyanov, V. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2009 · Cited by 617
Emotions in Multiple Languages
Dewaele, J.-M. (2010) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230289505 · Cited by 612
Cited by
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification
Barbieri, F., Camacho-Collados, J., Espinosa Anke, L., & Neves, L. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.148 · Cited by 674
A Dataset of Hindi-English Code-Mixed Social Media Text for Hate Speech Detection
Bohra, A., Vijay, D., Singh, V., Akhtar, S. S., & Shrivastava, M. (2018) · Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media · doi:10.18653/v1/w18-1105 · Cited by 265
Bhuiyan, M. H. M., Kumar, M. K. B., & Staicu, C.-A. (2026) · Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering · doi:10.1145/3744916.3787766 · Cited by 5
Metadata
- Authors
- Barman, U., Das, A., Wagner, J., & Foster, J.
- Venue
- Proceedings of the First Workshop on Computational Approaches to Code Switching
- Published
- 2014
- Pages
- 13-23
- OpenAlex
- W2250548009
- Cited by
- 267
- References
- 30
- FWCI
- 10.87
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Code Mixing: A Challenge for Language Identification in the Language of Social Media. https://app.cochat.ai/papers/10.3115/v1/w14-3902
- The paper
- Barman, U., Das, A., Wagner, J., & Foster, J. (2014). Code Mixing: A Challenge for Language Identification in the Language of Social Media. Proceedings of the First Workshop on Computational Approaches to Code Switching. https://doi.org/10.3115/v1/w14-3902
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.