Social Bias Frames: Reasoning about Social and Power Implications of Language
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
contains content that may be offensive or upsetting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Automatic acquisition of hyponyms from large text corpora
Hearst, M. A. (1992) · Proceedings of the 14th conference on Computational linguistics - · doi:10.3115/992133.992154 · Cited by 3,253
Lakoff, R. (1973) · Language in Society · doi:10.1017/s0047404500000051 · Cited by 3,138
From Word Embeddings To Document Distances
Kusner, M. J., Sun, Y., Kolkin, N., & Weinberger, K. Q. (2015) · PolyPublie (École Polytechnique de Montréal) · Cited by 2,700
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395
Liu, C.-W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1230 · Cited by 1,228
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Cited by
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
“…(2019); Sweeney and Najafian (2019); Vaidya et al. (2019); Gaut et al. (2020); Gencoglu (2020); Hovy et al. (2020); Hutchinson et al. (2020); Kim et al. (2020); Peng et al. (2020); Rios (2020); Sap et al. (2020); Shah et al. (2020); Sheng et al. (2020); Tan et al. (2020); Zhang et al. (2020a,b).”
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Self-citationCan Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Self-citationGender bias and stereotypes in Large Language Models
Kotek, H., Dockum, R., & Sun, D. (2023) · Proceedings of The ACM Collective Intelligence Conference · doi:10.1145/3582269.3615599 · Cited by 381
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., & Kamar, E. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.234 · Cited by 279
Self-citationA Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support
Sharma, A., Miner, A., Atkins, D., & Althoff, T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.425 · Cited by 268
BBQ: A hand-built bias benchmark for question answering
Parrish, A., Chen, A., Nangia, N., Padmakumar, V., Phang, J., Thompson, J., Htut, P. M., & Bowman, S. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.165 · Cited by 186
Social chemistry 101: Learning to reason about social and moral norms
Forbes, M., Hwang, J. D., Shwartz, V., Sap, M., & Choi, Y. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.48 · Cited by 169
Self-citationDankwa-Mullan, I., Scheufele, E. L., Matheny, M. E., Quintana, Y., Chapman, W. W., Jackson, G., & South, B. R. (2021) · Journal of Health Care for the Poor and Underserved · doi:10.1353/hpu.2021.0065 · Cited by 108
Challenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112
ProsocialDialog: A prosocial backbone for conversational agents
Kim, H., Yu, Y., Jiang, L., Lu, X., Khashabi, D., Kim, G., Choi, Y., & Sap, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.267 · Cited by 80
Self-citationChoi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · arXiv · doi:10.48550/arxiv.2305.14938 · Cited by 73
Quantifying and alleviating political bias in language models
Liu, R., Jia, C., Wei, J., Xu, G., & Vosoughi, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103654 · Cited by 68
Knowledge of cultural moral norms in large language models
Ramezani, A., & Xu, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.26 · Cited by 54
Curto, G., Jojoa Acosta, M. F., Comim, F., & Garcia-Zapirain, B. (2024) · AI & SOCIETY · doi:10.1007/s00146-022-01494-z · Cited by 53
Theories of ”gender” in NLP bias research
Devinney, H., Björklund, J., & Björklund, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534627 · Cited by 49
Towards Measuring the Representation of Subjective Global Opinions in Language Models
Durmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.16388 · Cited by 48
Unveiling the Implicit Toxicity in Large Language Models
Wen, J., Ke, P., Sun, H., Zhang, Z., Li, C., Bai, J., & Huang, M. (2023) · arXiv · doi:10.48550/arxiv.2311.17391 · Cited by 40
Choi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.699 · Cited by 38
Ethical and methodological challenges in building morally informed AI systems
Hagendorff, T., & Danks, D. (2023) · AI and Ethics · doi:10.1007/s43681-022-00188-y · Cited by 36
Multimodal large language models for inclusive collaboration learning tasks
Lewis, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop · doi:10.18653/v1/2022.naacl-srw.26 · Cited by 30
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv · doi:10.48550/arxiv.2306.11932 · Cited by 20
“…LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and Ribeiro, 2022), so called “hallucinations” (LLMs making things up when…”
BackgroundKoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications
Lee, H., Hong, S., Park, J., Kim, T., Kim, G., & Ha, J.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track) · doi:10.18653/v1/2023.acl-industry.21 · Cited by 15
Debiasing pretrained text encoders by paying attention to paying attention
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.651 · Cited by 14
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Fleisig, E., Amstutz, A., Atalla, C., Blodgett, S. L., Daumé III, H., Olteanu, A., Sheng, E., Vann, D., & Wallach, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.343 · Cited by 14
Metadata
- Authors
- Sap, M., Gabriel, S., Qin, L., Jurafsky, D., Smith, N. A., & Choi, Y.
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 5477-5490
- OpenAlex
- W2984662393
- Cited by
- 332
- References
- 54
- FWCI
- 292.36
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.