CoChat
Sign Up
PreprintCited by 8Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Law, AI, and Intellectual Property

The 2024 Foundation Model Transparency Index

Open access
Bommasani, Rishi, Klyman, Kevin, Kapoor, Sayash, Longpre, Shayne, Xiong, Betty, Maslej, Nestor, Liang, Percy·2024·arXiv
DOI 10.48550/arxiv.2407.12929arXiv 2407.12929
Open this paper in CoChat — free accountOpen PDF

Abstract

Foundation models are increasingly consequential yet extremely opaque. To characterize the status quo, the Foundation Model Transparency Index (FMTI) was launched in October 2023 to measure the transparency of leading foundation model developers. FMTI 2023 assessed 10 major foundation model developers (e.g. OpenAI, Google) on 100 transparency indicators (e.g. does the developer disclose the wages it pays for data labor?). At the time, developers publicly disclosed very limited information with the average score being 37 out of 100. To understand how the status quo has changed, we conduct a follow-up study after 6 months: we score 14 developers against the same 100 indicators. While in FMTI 2023 we searched for publicly available information, in FMTI 2024 developers submit reports on the 100 transparency indicators, potentially including information that was not previously public. We find that developers now score 58 out of 100 on average, a 21 point improvement over FMTI 2023. Much of this increase is driven by developers disclosing information during the FMTI 2024 process: on average, developers disclosed information related to 16.6 indicators that was not previously public. We observe regions of sustained (i.e. across 2023 and 2024) and systemic (i.e. across most or all developers) opacity such as on copyright status, data access, data labor, and downstream impact. We publish transparency reports for each developer that consolidate information disclosures: these reports are based on the information disclosed to us via developers. Our findings demonstrate that transparency can be improved in this nascent ecosystem, the Foundation Model Transparency Index likely contributes to these improvements, and policymakers should consider interventions in areas where transparency has not improved.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2407.12929

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Foundation Model Transparency Index scores developers across upstream resources including data, data labor, and compute, and developers score worst precisely on those upstream indicators, even when they score well on downstream access.

    “The Foundation Model Transparency Index of Bommasani et al. (2023) and its 2024 successor Bommasani et al. (2024) score developers across upstream resources including data, data labor, and compute, and the empirically telling result is that developers score worst precisely on those upstream indicators, the commons-relevant layers of the stack, even when they score well on downstream access.”

    From Commons-Governed Artificial Intelligence: A Taxonomy of Collective Governance (2026) · cites this paper as Bommasani et al. (2024)

    The indicator-level assessment lists data, data labor, and compute as upstream subdomains. The standalone results (§4.1; page unavailable) report lower overall upstream scores (46%) than downstream scores (65%) and say developers disclose less about data, labor, and compute than about model evaluation or distribution. Those results support the broad contrast, but do not establish that developers who score poorly upstream score well specifically on downstream access. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • The transparency of the upstream resources that produced a model-its data, its data labor, and its computational cost-is a scarce and contested good.

    “The transparency of the upstream resources that produced a model, its data, its data labor, and its computational cost, is itself a scarce and contested good, as the low scores on exactly those indicators in the Foundation Model Transparency Index demonstrate (Bommasani et al., 2023(Bommasani et al., , 2024)).”

    From Commons-Governed Artificial Intelligence: A Taxonomy of Collective Governance (2026) · cites this paper as Bommasani et al., 2024

  • The defense against openwashing is structural: a model commons must be scored on the upstream indicators where the Foundation Model Transparency Index shows developers scoring worst, and openness must be treated as the entry condition to the commons rather than as its achievement.

    “The defense is structural rather than rhetorical, the insistence that a model commons be scored on the upstream indicators where the Foundation Model Transparency Index shows developers scoring worst (Bommasani et al., 2024), and that openness be treated as the entry condition to the commons rather than as its achievement.”

    From Commons-Governed Artificial Intelligence: A Taxonomy of Collective Governance (2026) · cites this paper as Bommasani et al., 2024

  • The monitoring column is strong across the compute and model rows, because compute is intrinsically measurable and because transparency instruments exist for models, but it is only emerging on the energy row, where carbon accounting is recent.

    “The monitoring column is strong across the compute and model rows, because compute is intrinsically measurable (Sastry et al., 2024) and because transparency instruments exist for models (Bommasani et al., 2024;Mitchell et al., 2019), but it is only emerging on the energy row, where carbon accounting is recent.”

    From Commons-Governed Artificial Intelligence: A Taxonomy of Collective Governance (2026) · cites this paper as Bommasani et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 8

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 67

  1. We provided guidance that the flagship foundation model should be "based on a combination of the following factors: greatest resource expenditure, most advanced capabilities, and greatest societal impact." 8 Cohere and Inflection declined to participate in FMTI v1.1. References Emmanuel Alloa. Transparency: A Magic Concept of Modernity, pp. 21-55. Springer International Publishing, Cham, 2018. ISBN 978-3-319-77161-8. doi: 10.1007/978-3-319-77161-8_3. URL https://doi.org/10. 1007/978-3-319-77161-8_3. · DOI
  2. Emmanuel Alloa and Dieter Thomä. Transparency: Thinking Through an Opaque Concept, pp. 1-13. 06 2018. ISBN 978-3-319-77160-1. doi: 10.1007/978-3-319-77161-8_1. · DOI
  3. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. The falcon series of open language models. arXiv preprint arXiv:2311.16867, 2023. Published in Transactions on Machine Learning Research (12/2024) · arXiv
  4. Mike Ananny and Kate Crawford. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3):973-989, 2018a. doi: 10.1177/ 1461444816676645. URL https://doi.org/10.1177/1461444816676645. · DOI · Paper page
  5. Mike Ananny and Kate Crawford. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3):973-989, 2018b. doi: 10.1177/ 1461444816676645. URL https://doi.org/10.1177/1461444816676645. · DOI · Paper page
  6. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. · arXiv
  7. Jack Bandy and Nicholas Vincent. Addressing "documentation debt" in machine learning research: A retrospective datasheet for bookcorpus. arXiv preprint arXiv:2105.05241, 2021. · arXiv
  8. Emily M. Bender and Batya Friedman. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6: 587-604, 2018. doi: 10.1162/tacl_a_00041. URL https://aclanthology.org/Q18-1041. · DOI · Paper page
  9. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
  10. Clare Birchall. Radical transparency? Cultural Studies and Critical Methodologies, 14(1):77-88, 2014. doi: 10.1177/1532708613517442. URL https://doi.org/10.1177/1532708613517442. · DOI · Paper page
  11. Clare Birchall. Radical secrecy: The ends of transparency in datafied America, volume 60. U of Minnesota Press, 2021.
  12. Sid Black, Stella Rose Biderman, Eric Hallahan, Quentin G. Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, M. Pieler, Usvsn Sai Prashanth, Shivanshu Purohit, Laria Reynolds, J. Tow, Ben Wang, and Samuel Weinbach. GPT-NeoX-20B: An open-source autoregressive language model. arXiv, 2022.
  13. Robert Bloomfield and Maureen O'Hara. Market transparency: Who wins and who loses? The Review of Financial Studies, 12(1):5-35, 1999. ISSN 08939454, 14657368. URL http://www.jstor.org/stable/ 2645985.
  14. Michael D Boldin. A critique of the traditional composite index methodology. Journal of economic and social measurement, 25(3-4):119-140, 1999.
  15. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dorottya Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. · arXiv
  16. Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, and Percy Liang. The foundation model transparency index. ArXiv, abs/2310.12941, 2023a. URL https://api.semanticscholar.org/CorpusID:264306385. · arXiv
  17. Rishi Bommasani, Dilara Soylu, Thomas Liao, Kathleen A. Creel, and Percy Liang. Ecosystem graphs: The social footprint of foundation models. ArXiv, abs/2303.15772, 2023b. URL https://api.semanticscholar. org/CorpusID:257771875. · arXiv
  18. Rishi Bommasani, Kevin Klyman, Shayne Longpre, Betty Xiong, Sayash Kapoor, Nestor Maslej, Arvind Narayanan, and Percy Liang. Foundation model transparency reports. ArXiv, abs/2402.16268, 2024. URL https://api.semanticscholar.org/CorpusID:267938721. · arXiv
  19. Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. What does it mean for a language model to preserve privacy? In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 2280-2292, 2022.
  20. Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, et al. Toward trustworthy ai development: mechanisms for supporting verifiable claims. arXiv preprint arXiv:2004.07213, 2020. · arXiv
  21. Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency, pp. 77-91, 2018.
  22. Rosario Cammarota, Matthias Schunter, Anand Rajan, Fabian Boemer, Ágnes Kiss, Amos Treiber, Christian Weinert, Thomas Schneider, Emmanuel Stapf, Ahmad-Reza Sadeghi, et al. Trustworthy ai inference systems: An industry research view. arXiv preprint arXiv:2008.04449, 2020. · arXiv
  23. Eric Corbett and Emily Denton. Interrogating the t in facct. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 1624-1634, 2023.
  24. Kate Crawford. The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press, 2021.
  25. DeepSeek-AI, :, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, A. X. Liu, Bo Liu, Wen Liu, Xiaodong Liu, Xin Liu, Yiyuan Liu, Haoyu Lu, Shanghao Lu, Fuli Luo, Shirong Ma, Xiaotao Nie, Tian Pei, Yishi Piao, Junjie Qiu, Hui Qu, Tongzheng Ren, Zehui Ren, Chong Ruan, Zhangli Sha, Zhihong Shao, Junxiao Song, Xuecheng Su, Jingxiang Sun, Yaofeng Sun, Minghui Tang, Bingxuan Wang, Peiyi Wang, Shiyu Wang, Yaohui Wang, Yongji Wang, Tong Wu, Y. Wu, Xin Xie, Zhenda Xie, Ziwei Xie, Yiliang Xiong, Hanwei Xu, R. X. Xu, Yanhong Xu, Dejian Yang, Yuxiang You, Shuiping Yu, Xingkai Yu, B. Zhang, Haowei Zhang, Lecong Zhang, Liyue Zhang, Mingchuan Zhang, Minghua Zhang, Wentao Zhang, Yichao Zhang, Chenggang Zhao, Yao Zhao, Shangyan Zhou, Shunfeng Zhou, Qihao Zhu, and Yuheng Zou. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024. · arXiv
  26. EU. Official journal of the european union 2016. Official Journal of the European Union, L 119/1, Apr 2016. URL https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1552662547490&uri=CELEX% 3A32016R0679.
  27. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Ill, and Kate Crawford. Datasheets for datasets. arXiv preprint arXiv:1803.09010, 2018. · arXiv
  28. Ritwick Ghosh and Hilary Oliva Faxon. Smart corruption: Satirical strategies for gaming accountability. Big Data & Society, 10(1):20539517231164119, 2023. doi: 10.1177/20539517231164119. URL https: //doi.org/10.1177/20539517231164119. · DOI
  29. Charles AE Goodhart. Problems of monetary management: the uk experience. In Monetary theory and practice, pp. 91-121. Springer, 1984.
  30. Nelson Granados and Alok Gupta. Transparency strategy: Competing with information in a digital world. MIS Quarterly, 37(2):637-641, 2013. ISSN 02767783. URL http://www.jstor.org/stable/43825928.
  31. Salvatore Greco, Alessio Ishizaka, Menelaos Tasiou, and Gianpiero Torrisi. On the methodological framework of composite indices: A review of the issues of weighting, aggregation, and robustness. Social Indicators Research, 141:1-34, 01 2019. doi: 10.1007/s11205-017-1832-9. · DOI · Paper page
  32. Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, and Hannaneh Hajishirzi. Olmo: Accelerating the science of language models. arXiv preprint arXiv:2402.00838, 2024. · arXiv
  33. Byung-Chul Han. The transparency society. Stanford University Press, 2015. Published in Transactions on Machine Learning Research (12/2024)
  34. Lauren Kogen. From statistics to stories: Indices and indicators as communication tools for social change. The International Journal of Press/Politics, 0(0):19401612221094246, 2022. doi: 10.1177/19401612221094246. URL https://doi.org/10.1177/19401612221094246. · DOI
  35. Noam Kolt, Markus Anderljung, Joslyn Barnhart, Asher Brass, Kevin M. Esvelt, Gillian K. Hadfield, Lennart Heim, Mikel Rodriguez, Jonas B. Sandbrink, and Thomas Woodside. Responsible reporting for frontier ai development. 2024. URL https://api.semanticscholar.org/CorpusID:268875838.
  36. Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. Language generation models can cause harm: So what can we do about it? an actionable survey. arXiv preprint arXiv:2210.07700, 2022. · arXiv
  37. Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700, 2019. · arXiv
  38. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787, 2024. · arXiv
  39. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. Holistic evaluation of language models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=iO4LZibEqW. Featured Certification, Expert Certification.
  40. Yeyi Liu, Martin Heinberg, Xuan Huang, and Andreas B. Eisingerich. Building a competitive advantage based on transparency: When and why does transparency matter for corporate social responsibility? Business Horizons, 66(4):517-527, 2023. ISSN 0007-6813. doi: https://doi.org/10.1016/j.bushor.2022.10.004. URL https://www.sciencedirect.com/science/article/pii/S0007681322001306. · DOI
  41. Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, et al. The data provenance initiative: A large scale audit of dataset licensing & attribution in ai. arXiv preprint arXiv:2310.16787, 2023a. · arXiv
  42. Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, et al. A pretrainer's guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. arXiv preprint arXiv:2305.13169, 2023b. · arXiv
  43. Shayne Longpre, Sayash Kapoor, Kevin Klyman, Ashwin Ramaswami, Rishi Bommasani, Borhane Blili- Hamelin, Yangsibo Huang, Aviya Skowron, Zheng-Xin Yong, Suhas Kotha, et al. A safe harbor for ai evaluation and red teaming. arXiv preprint arXiv:2403.04893, 2024a. · arXiv
  44. Alexandra Sasha Luccioni and Alex Hernández-García. Counting carbon: A survey of factors influencing the emissions of machine learning. ArXiv, abs/2302.08476, 2023. · arXiv
  45. Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, Anton Belyi, Haotian Zhang, Karanjeet Singh, Doug Kang, Ankur Jain, Hongyu Hè, Max Schwarzer, Tom Gunter, Xiang Kong, Aonan Zhang, Jianyu Wang, Chong Wang, Nan Du, Tao Lei, Sam Wiseman, Guoli Yin, Mark Lee, Zirui Wang, Ruoming Pang, Peter Grasch, Alexander Toshev, and Yinfei Yang. Mm1: Methods, analysis & insights from multimodal llm pre-training. arXiv preprint arXiv:2403.09611, 2024. · arXiv
  46. Luke Merrick, Danmei Xu, Gaurav Nuti, and Daniel Campos. Arctic-embed: Scalable, efficient, and accurate text embedding models. arXiv preprint arXiv:2405.05374, 2024. · arXiv
  47. Brent Mittelstadt. Principles alone cannot guarantee ethical ai. Nature Machine Intelligence, 1(11):501-507, November 2019. ISSN 2522-5839. doi: 10.1038/s42256-019-0114-4. URL https://doi.org/10.1038/ s42256-019-0114-4. · DOI · Paper page
  48. Shakir Mohamed, Marie-Therese Png, and William Isaac. Decolonial ai: Decolonial theory as sociotechnical foresight in artificial intelligence. Philosophy & Technology, 33(4):659-684, December 2020. ISSN 2210-5441. doi: 10.1007/s13347-020-00405-8. URL https://doi.org/10.1007/s13347-020-00405-8. · DOI · Paper page
  49. David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350, 2021. · arXiv
  50. Inioluwa Deborah Raji and Joy Buolamwini. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES '19, pp. 429-435, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450363242. doi: 10.1145/3306618.3314244. URL https://doi.org/10. 1145/3306618.3314244. · DOI · Paper page
  51. Ambuj D Sagar and Adil Najam. The human development index: a critical review. Ecological economics, 25 (3):249-264, 1998.
  52. Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. "Everyone wants to do the model work, not the data work": Data cascades in high-stakes AI. In CHI, CHI '21, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450380966. doi: 10.1145/3411764.3445518. URL https://doi.org/10.1145/3411764.3445518. · DOI · Paper page
  53. Fabio Gaetano Santeramo. Methodological challenges in building composite indexes: Linking theory to practice. In Emerging trends in the development and application of composite indicators, pp. 127-139. IGI Global, 2017.
  54. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36, 2024.
  55. Martin Schlossarek, Miroslav Syrovátka, and Ondřej Vencálek. The importance of variables in composite indices: A contribution to the methodology and application to development indices. Social Indicators Research, 145(3):1125-1160, 2019.
  56. Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. Green ai. Communications of the ACM, 63 (12):54-63, 2020.
  57. Irene Solaiman. The gradient of generative ai release: Methods and considerations. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 111-122, 2023.
  58. Daniel J. Solove. Artificial intelligence and privacy. Florida Law Review, 2024. doi: 10.2139/ssrn.4713111. URL https://ssrn.com/abstract=4713111. Forthcoming January 2025. · DOI · Paper page
  59. Reka Team, Aitor Ormazabal, Che Zheng, Cyprien de Masson d'Autume, Dani Yogatama, Deyu Fu, Donovan Ong, Eric Chen, Eugenie Lamprecht, Hai Pham, Isaac Ong, Kaloyan Aleksiev, Lei Li, Matthew Henderson, Max Bain, Mikel Artetxe, Nishant Relan, Piotr Padlewski, Qi Liu, Ren Chen, Samuel Phua, Yazheng Yang, Yi Tay, Yuqi Wang, Zhongkai Zhu, and Zhihui Xie. Reka core, flash, and edge: A series of powerful multimodal language models. arXiv preprint arXiv:2404.12387, 2024. · arXiv
  60. Jorge I. Valdovinos. Transparency as ideology, ideology as transparency: Towards a critique of the meta-aesthetics of neoliberal hegemony. Open Cultural Studies, 2(1):654-667, 2018. doi: doi:10.1515/ culture-2018-0059. URL https://doi.org/10.1515/culture-2018-0059. · DOI
  61. Lee Vinsel. Moving violations: automobiles, experts, and regulations in the United States. JHU Press, 2019.
  62. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359, 2021. · arXiv
  63. Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos- Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, and William S. Isaac. Sociotechnical safety evaluation of generative ai systems. 2023. URL https://arxiv. org/abs/2310.11986.
  64. Amy Winograd. Loose-lipped large language modells spill your secrets: The privacy implications of large language models. Harvard Journal of Law and Technology, 36(2), 2023.
  65. Yue Wu, Kaifu Zhang, and Jinhong Xie. Bad greenwashing, good greenwashing: Corporate social responsibility and information transparency. Management Science, 66(7):3095-3112, 2020.
  66. Ellen Pei-Yi Yu, Christine Qian Guo, and Bac Van Luu. Environmental, social and governance transparency and firm value. Business Strategy and the Environment, 27(7):987-1004, 2018.
  67. Monika Zalnieriute. "transparency-washing" in the digital age : A corporate agenda of procedural fetishism. Technical report, 2021. URL http://hdl.handle.net/11159/468588.

Authors
Bommasani, R., Klyman, K., Kapoor, S., Longpre, S., Xiong, B., Maslej, N., & Liang, P.
Affiliations
  • Stanford University (US)
  • Princeton University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2407.12929
arXiv
2407.12929
OpenAlex
W4406058595
Cited by
8
References
0
Citations by year
2026: 3 · 2025: 5
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
foundation models, AI transparency, transparency indicators, AI developers, data labor, data access, downstream effects, transparency reports, AI governance, OpenAI, Google

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review