The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Open accessAbstract
We present the results and the main findings of SemEval-2019 Task 6 on Identifying and Categorizing Offensive Language in Social Media (OffensEval).The task was based on a new dataset, the Offensive Language Identification Dataset (OLID), which contains over 14,000 English tweets.It featured three sub-tasks.In sub-task A, the goal was to discriminate between offensive and non-offensive posts.In sub-task B, the focus was on the type of offensive content in the post.Finally, in sub-task C, systems had to detect the target of the offensive posts.OffensEval attracted a large number of participants and it was one of the most popular tasks in SemEval-2019.In total, about 800 teams signed up to participate in the task, and 115 of them submitted results, which we present and analyze in this report.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Predicting the Type and Target of Offensive Posts in Social Media
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1144 · Cited by 720
Self-citationHate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742760 · Cited by 712
Burnap, P., & Williams, M. L. (2015) · Policy & Internet · doi:10.1002/poi3.85 · Cited by 616
Modeling the Detection of Textual Cyberbullying
Dinakar, K., Reichart, R., & Lieberman, H. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i3.14209 · Cited by 512
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2010) · doi:10.21236/ada547371 · Cited by 491
Understanding Abuse: A Typology of Abusive Language Detection Subtasks
Waseem, Z., Davidson, T., Warmsley, D., & Weber, I. (2017) · Proceedings of the First Workshop on Abusive Language Online · doi:10.18653/v1/w17-3012 · Cited by 474
Benchmarking Aggression Identification in Social Media.
Kumar, R., Ojha, A. K., Malmasi, S., & Zampieri, M. (2018) · International Conference on Computational Linguistics · Cited by 474
Self-citationLocate the Hate: Detecting Tweets against Blacks
Kwok, I., & Wang, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8539 · Cited by 446
Improving Cyberbullying Detection with User Context
Dadvar, M., Trieschnigg, D., Ordelman, R., & de Jong, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-36973-5_62 · Cited by 376
Learning from Bullying Traces in Social Media
Xu, J., Jun, K.-S., Zhu, X., & Bellmore, A. D. (2012) · Cited by 332
Detecting Hate Speech in Social Media
Harvard Medical School, B., Malmasi, S., Zampieri, M., & University of Wolverhampton, W. (2017) · RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning · doi:10.26615/978-954-452-049-6_062 · Cited by 285
Self-citationChallenges in discriminating profanity from hate speech
Malmasi, S., & Zampieri, M. (2018) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2017.1409284 · Cited by 264
Self-citationOverview of the GermEval 2018 Shared Task on the Identification of Offensive Language
Wiegand, M., Siegel, M., & Ruppenhofer, J. (2019) · Publication Server of the Institute for German Language (Institute for German Language) · Cited by 232
Convolutional Neural Networks for Toxic Comment Classification
Georgakopoulos, S. V., Tasoulis, S. K., Vrahatis, A. G., & Plagianakos, V. P. (2018) · Proceedings of the 10th Hellenic Conference on Artificial Intelligence · doi:10.1145/3200947.3208069 · Cited by 194
Godin, F., Vandersmissen, B., De Neve, W., & Van de Walle, R. (2015) · Proceedings of the Workshop on Noisy User-generated Text · doi:10.18653/v1/w15-4322 · Cited by 188
Liu, P., Li, W., & Zou, L. (2019) · Proceedings of the 13th International Workshop on Semantic Evaluation · doi:10.18653/v1/s19-2011 · Cited by 171
Aroyehun, S. T., & Gelbukh, A. F. (2018) · International Conference on Computational Linguistics · Cited by 116
Nikolov-Radivchev at SemEval-2019 Task 6: Offensive Tweet Classification with BERT and Ensembles
Nikolov, A., & Radivchev, V. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2123 · Cited by 72
Pavlopoulos, J., Thain, N., Dixon, L., & Androutsopoulos, I. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2102 · Cited by 53
UM-IU@LING at SemEval-2019 Task 6: Identifying Offensive Tweets Using BERT and SVMs
Zhu, J. Y., Tian, Z., & Kübler, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2138 · Cited by 43
MIDAS at SemEval-2019 Task 6: Identifying Offensive Posts and Targeted Offense from Twitter
Mahata, D., Zhang, H., Uppal, K., Kumar, Y., Shah, R. R., Shahid, S., Mehnaz, L., & Anand, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2122 · Cited by 31
Merging Datasets for Aggressive Text Identification
Fortuna, P., Ferreira, J. V., Pires, L., Routar, G., & Nunes, S. (2018) · International Conference on Computational Linguistics · Cited by 30
LTL-UDE at SemEval-2019 Task 6: BERT and Two-Vote Classification for Categorizing Offensiveness
Aggarwal, P., Horsmann, T., Wojatzki, M., & Zesch, T. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2121 · Cited by 22
Embeddia at SemEval-2019 Task 6: Detecting Hate with Neural Network and Transfer Learning Approaches
Pelicon, A., Martinc, M., & Novak, P. K. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2108 · Cited by 21
Seganti, A., Sobol, H., Orlova, I., Kim, H., Staniszewski, J., Krumholc, T., & Koziel, K. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2126 · Cited by 20
Amobee at SemEval-2019 Tasks 5 and 6: Multiple Choice CNN Over Contextual Embedding
Rozental, A., & Biton, D. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2066 · Cited by 20
jhan014 at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media
Han, J., Wu, S., & Liu, X. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2116 · Cited by 18
nlpUP at SemEval-2019 Task 6: A Deep Neural Language Model for Offensive Language Detection
Mitrović, J., Birkeneder, B., & Granitzer, M. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2127 · Cited by 18
BNU-HKBU UIC NLP Team 2 at SemEval-2019 Task 6: Detecting Offensive Language Using BERT model
Wu, Z., Zheng, H., Wang, J., Su, W., & Fong, J. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2099 · Cited by 17
Plaza-del-Arco, F. M., Molina-González, M. D., Martín, M., & López, L. A. U. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2129 · Cited by 16
Wikipedia Talk Labels: Toxicity
Thain, N., Dixon, L., & Wulczyn, E. (2017) · Figshare · doi:10.6084/m9.figshare.4563973 · Cited by 16
Duluth at SemEval-2019 Task 6: Lexical Approaches to Identify and Categorize Offensive Tweets
Ted Pedersen, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2106 · Cited by 15
KMI-Coling at SemEval-2019 Task 6: Exploring N-grams for Offensive Language detection
Rani, P., & Ojha, A. K. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2119 · Cited by 15
UVA Wahoos at SemEval-2019 Task 6: Hate Speech Identification using Ensemble Machine Learning
Ramakrishnan, M., Zadrozny, W., & Tabari, N. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2141 · Cited by 12
Altın, L. S. M., Serrano, À. B., & Saggion, H. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2120 · Cited by 12
Wiedemann, G., Ruppert, E., & Biemann, C. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2137 · Cited by 12
Bidirectional Recurrent Models for Offensive Tweet ClassificationPreprint
Cambray, A., & Podsadowski, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08808 · Cited by 11
Sridharan, M., & Swapna, T. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2097 · Cited by 9
Suseelan, A., Sivanaiah, R., Logesh, B., Harshini, S., Geetika, B. S. R., Dyaneswaran, S., Rajendram, S. M., & Mirnalinee, T. T. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2132 · Cited by 9
Aglionby, G., Davis, C., Mishra, P., Caines, A., Yannakoudakis, H., Rei, M., Shutova, E., & Buttery, P. J. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2100 · Cited by 9
Kebriaei, E., Karimi, S., Sabri, N., & Shakery, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2107 · Cited by 9
Indurthi, V., Syed, B., Shrivastava, M., Gupta, M., & Varma, V. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2109 · Cited by 9
Fortuna, P., & Nunes, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2131 · Cited by 8
Wang, B., Zhou, X., & Zhang, X. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2143 · Cited by 8
An Exploration of State-of-the-art Methods for Offensive Language DetectionPreprint
Uglow, H., Zlocha, M., & Zmyslony, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07445 · Cited by 7
UBC-NLP at SemEval-2019 Task 6: Ensemble Learning of Offensive Content With Enhanced Training Data
Rajendran, A. K., Zhang, C., & Abdul-Mageed, M. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2136 · Cited by 7
Oberstraß, A., Romberg, J., Stoll, A., & Conrad, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2112 · Cited by 7
Singh, P., & Chand, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2128 · Cited by 7
Puiu, A.-B., & Brabete, A.-O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00665 · Cited by 5
la Peña, G. L. D., & Rosso, P. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2104 · Cited by 5
Zhang, Y., Xu, B., & Zhao, T. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2101 · Cited by 5
Swamy, S. D., Jamatia, A., Gambäck, B., & Das, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2124 · Cited by 5
USF at SemEval-2019 Task 6: Offensive Language Detection Using LSTM With Word Embeddings
Goel, B., & Sharma, R. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2139 · Cited by 5
YNU-HPCC at SemEval-2019 Task 6: Identifying and Categorising Offensive Language on Twitter
Zhou, C., Wang, J., & Zhang, X. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2142 · Cited by 4
JTML at SemEval-2019 Task 6: Offensive Tweets Identification using Convolutional Neural Networks
Torres, J., & Vaca, C. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2117 · Cited by 3
Mukherjee, P., Pal, M., Banerjee, S., & Naskar, S. K. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2118 · Cited by 3
bhanodaig at SemEval-2019 Task 6: Categorizing Offensive Language in social media
Kumar, R., Bhanodai, G., Pamula, R., & Chennuru, M. R. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2098 · Cited by 3
Self-citationEl-Zanaty, Z. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2144 · Cited by 2
TUVD team at SemEval-2019 Task 6: Offense Target Identification
Shushkevich, E., Cardiff, J., & Rosso, P. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2135 · Cited by 2
Kannan, M., & Stein, L. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2134 · Cited by 1
Cited by
Predicting the Type and Target of Offensive Posts in Social Media
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1144 · Cited by 720
Self-citationTweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification
Barbieri, F., Camacho-Collados, J., Espinosa Anke, L., & Neves, L. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.148 · Cited by 674
Resources and benchmark corpora for hate speech detection: a systematic review
Poletto, F., Basile, V., Sanguinetti, M., Bosco, C., & Patti, V. (2021) · Language Resources and Evaluation · doi:10.1007/s10579-020-09502-8 · Cited by 385
Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z., & Çöltekin, Ç. (2020) · Proceedings of the Fourteenth Workshop on Semantic Evaluation · doi:10.18653/v1/2020.semeval-1.188 · Cited by 378
Self-citationWeight Poisoning Attacks on Pretrained Models
Kurita, K., Michel, P., & Neubig, G. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.249 · Cited by 354
Metadata
- Authors
- Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R.
- Venue
- Proceedings of the 13th International Workshop on Semantic Evaluation
- Published
- 2019
- Pages
- 75-86
- OpenAlex
- W2922580172
- Cited by
- 701
- References
- 67
- FWCI
- 46.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.