CoChat
Sign Up
Cited by 58Digitalization, Law, and RegulationEthics and Social Impacts of AILaw, AI, and Intellectual Property

Addressing the regulatory gap: moving towards an EU AI audit ecosystem beyond the AI Act by including civil society

Open access
Hartmann, David, de Pereira, José Renato Laranjeira, Streitbörger, Chiara, Berendt, Bettina·2025·AI and Ethics
DOI 10.1007/s43681-024-00595-3
Open this paper in CoChat — free accountOpen PDF

Abstract

The European legislature has proposed the Digital Services Act (DSA) and Artificial Intelligence Act (AIA) to regulate platforms and Artificial Intelligence (AI) products. We review to what extent third-party audits are part of both laws and how is access to information on models and the data provided. By considering the value of third-party audits and third-party data access in an audit ecosystem, we identify a regulatory gap in that the AIA does not provide access to data for researchers and civil society. Our contributions to the literature include: (1) Defining an AI audit ecosystem incorporating compliance and oversight. (2) Highlighting a regulatory gap within the DSA and AIA regulatory framework, preventing the establishment of an AI audit ecosystem that has effective oversight by civil society and academia. (3) Emphasizing that third-party audits by research and civil society must be part of that ecosystem, we call for AIA amendments and delegated acts to include data and model access for certain AI products. Furthermore, we call for the DSA to provide NGOs and investigative journalists with data access to platforms by delegated acts and for adaptions and amendments of the AIA to provide third-party audits and data and model access, at least for high-risk systems. Regulations modeled after EU AI regulations should enable data access and third-party audits, fostering an AI audit ecosystem that promotes compliance and oversight mechanisms.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s43681-024-00595-3

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Few studies have explored the relationship between content moderation and echo chambers due to restricted access to platform data.

    “Despite the critical role of content moderation, few studies have explored its relationship with echo chambers due to restricted access to platform data about how content moderation works and how content is deleted and recommended [88].”

    From A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes (2025) · cites this paper as [88]

    Section 4.1 describes restrictions on researchers’ access to platform data, and “Information and Power Asymmetries” links limited transparency to difficulties evaluating content moderation (page numbers unavailable). Neither passage says that few studies examine the relationship between content moderation and echo chambers, or that data-access restrictions explain a shortage of such studies. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • The EU DSA grants vetted researchers access to platform data and enables recommendation system audits.

    “which grants vetted researchers access to platform data and enables recommendation system audits [88].”

    From A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes (2025) · cites this paper as [88]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 56

  1. Bazerman, M.H., Morgan, K.P., Loewenstein, G.F., et al.: The impossibil- ity of auditor independence. Sloan management review 38, 89-94 (1997)
  2. Bieker, F., Friedewald, M., Hansen, M., Obersteller, H., Rost, M.: A Pro- cess for Data Protection Impact Assessment Under the European General Data Protection Regulation. In: Schiffner, S., Serna, J., Ikonomou, D., Rannenberg, K. (eds.) Privacy Technologies and Policy, pp. 21-37, Lec- ture Notes in Computer Science, Springer International Publishing, Cham (2016), ISBN 978-3-319-44760-5
  3. Birhane, A., Steed, R., Ojewale, V., Vecchione, B., Raji, I.D.: AI audit- ing: The Broken Bus on the Road to AI Accountability (Jan 2024), URL http://arxiv.org/abs/2401.14462, arXiv:2401.14462 [cs] · arXiv
  4. Buolamwini, J., Gebru, T.: Gender shades: Intersectional accuracy dis- parities in commercial gender classification. In: Conference on fairness, accountability and transparency, pp. 77-91, PMLR (2018)
  5. Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T.L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., Sharkey, L., Kr- ishna, S., Von Hagen, M., Alberti, S., Chan, A., Sun, Q., Gerovitch, M., Bau, D., Tegmark, M., Krueger, D., Hadfield-Menell, D.: Black- Box Access is Insufficient for Rigorous AI Audits (Jan 2024), URL http://arxiv.org/abs/2401.14446, arXiv:2401.14446 [cs] · arXiv
  6. Castets-Renard, C., Besse, P.: Ex ante accountability of the ai act: Between certification and standardization. Artificial Intelligence Law: Between Sec- toral Rules and Comprehensive Regime. Comparative Law Perspectives (Forthcoming), URL https://ssrn.com/abstract=4203925
  7. Chen, Y.J., Chiou, C.M., Huang, Y.W., Tu, P.W., Lee, Y.C., Chien, C.H.: A comparative study of medical device regulations:: Us, europe, canada, and taiwan. Therapeutic Innovation & Regulatory Science 52(1), 62- 69 (2018), URL https://doi.org/10.1177/2168479017716712, pMID: 29714608 · DOI
  8. Ebers, M., Hoch, V.R.S., Rosenkranz, F., Ruschemeier, H., Steinrötter, B.: The european commission's proposal for an artificial intelli- gence act-a critical assessment by members of the robotics and ai law society (rails). J 4(4), 589-603 (2021), ISSN 2571-8800, URL https://www.mdpi.com/2571-8800/4/4/43
  9. Floridi, L., Holweg, M., Taddeo, M., Amaya Silva, J., Mökander, J., Wen, Y.: capAI -A Procedure for Conducting Conformity Assessment of AI Systems in Line with the EU Artificial Intelligence Act (Mar 2022), URL https://papers.ssrn.com/abstract=4064091
  10. Gaddis, S.M.: An Introduction to Audit Studies in the Social Sciences (Aug 2017), URL https://papers.ssrn.com/abstract=3024262
  11. Globus-Harris, I., Kearns, M., Roth, A.: An Algorithmic Framework for Bias Bounties (May 2022), URL http://arxiv.org/abs/2201.10408, arXiv:2201.10408 [cs] · arXiv
  12. Goodman, E.P., Trehu, J.: AI Audit Washing and Accountability (Sep 2022), URL https://papers.ssrn.com/abstract=4227350
  13. Hansen, S., Loftus, J.: Model-Agnostic Auditing: A Lost Cause? Proceed- ings of the EWAF'23: European Workshop on Algorithmic Fairness (2023)
  14. Hickey, D., Schmitz, M., Fessler, D., Smaldino, P.E., Muric, G., Burghardt, K.: Auditing elon musk's impact on hate speech and bots. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 17, pp. 1133-1137 (2023)
  15. Kaesling: Art. 40. In: Hofmann, Raue (eds.) Digital Service Act, p. 33, Nomos (2023)
  16. Kazim, E., Denny, D.M., Koshiyama, A.: Ai auditing and impact assess- ment: According to the uk information commissioner's office. AI Ethics 1, 301-310 (2021), URL https://doi.org/10.1007/s43681-021-00039-2 · DOI
  17. Kop, M.: EU Artificial Intelligence Act: The European Approach to AI (Sep 2021), URL https://papers.ssrn.com/abstract=3930959
  18. Koshiyama, A., Kazim, E., Treleaven, P.: Algorithm auditing: Manag- ing the legal, ethical, and technological risks of artificial intelligence, ma- chine learning, and associated algorithms. Computer 55(4), 40-50 (2022), https://doi.org/10.1109/MC.2021.3067225 · DOI · Paper page
  19. Koshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., Adams, J., Hitrova, C., Barnett, J., Nachev, P., Barber, D., Chamorro-Premuzic, T., Klemmer, K., Gregorovic, M., Khan, S., Lomas, E.: Towards Algo- rithm Auditing: A Survey on Managing Legal, Ethical and Technologi- cal Risks of AI, ML and Associated Algorithms. SSRN (Jan 2021), URL https://papers.ssrn.com/abstract=3778998
  20. Laux, J., Wachter, S., Mittelstadt, B.: Taming the Few: Platform Regulation, Independent Audits, and the Risks of Capture Created by the DMA and DSA (Sep 2021), URL https://papers.ssrn.com/abstract=4096655
  21. Laux, J., Wachter, S., Mittelstadt, B.: Three Pathways for Standardisation and Ethical Disclosure by Default under the European Union Artificial Intelligence Act. SSRN Electronic Journal (2023), ISSN 1556-5068, URL https://www.ssrn.com/abstract=4365079
  22. Laux, J., Wachter, S., Mittelstadt, B.: Three pathways for stan- dardisation and ethical disclosure by default under the Eu- ropean Union Artificial Intelligence Act. Computer Law & Security Review 53, 105957 (2024), ISSN 0267-3649, URL https://www.sciencedirect.com/science/article/pii/S0267364924000244
  23. Li, R., Kingsley, S., Fan, C., Sinha, P., Wai, N., Lee, J., Shen, H., Eslami, M., Hong, J.: Participation and division of labor in user-driven algorithm audits: How do everyday users work together to surface algorithmic harms? CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023)
  24. Mancosu, M., Vegetti, F.: What you can scrape and what is right to scrape: A proposal for a tool to collect public facebook data. Social Media + Society 6(3), 2056305120940703 (2020), URL https://doi.org/10.1177/2056305120940703 · DOI · Paper page
  25. McFadden, M., Jones, K., Taylor, E., Osborn, G.: Harmonising Artificial Intelligence: the role of standards in the eu ai regulation. Oxford Internet Institute (2021)
  26. McNamara, A., Smith, J., Murphy-Hill, E.: Does ACM's code of ethics change ethical decision making in software development? In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 729-733, ACM, Lake Buena Vista FL USA (Oct 2018), ISBN 978-1- 4503-5573-5, URL https://dl.acm.org/doi/10.1145/3236024.3264833 · DOI · Paper page
  27. Metaxa, D., Park, J.S., Robertson, R.E., Karahalios, K., Wilson, C., Hancock, J., Sandvig, C.: Auditing algorithms: Understanding algorith- mic systems from the outside in. Foundations and Trends® in Hu- man-Computer Interaction 14(4), 272-344 (2021), ISSN 1551-3955, URL http://dx.doi.org/10.1561/1100000083 · DOI
  28. Munn, L.: The uselessness of AI ethics. AI and Ethics 3(3), 869-877 (Aug 2023), ISSN 2730-5953, 2730-5961, URL https://link.springer.com/10.1007/s43681-022-00209-w · DOI
  29. Mökander, J., Floridi, L.: Ethics-Based Auditing to Develop Trustworthy AI. Minds and Machines 31(2), 323-327 (Jun 2021), ISSN 1572-8641, URL https://doi.org/10.1007/s11023-021-09557-8 · DOI · Paper page
  30. Mökander, J., Schuett, J., Kirk, H.R., Floridi, L.: Auditing large language models: a three-layered approach (Feb 2023), URL http://arxiv.org/abs/2302.08500, arXiv:2302.08500 [cs] · arXiv
  31. Novelli, C., Taddeo, M., Floridi, L.: Accountability in artificial intelligence: What it is and how it works. Ai and Society: Knowledge, Culture and Communication pp. 1-12 (forthcoming)
  32. Ohme, J., Araujo, T., Boeschoten, L., Freelon, D., Ram, N., Reeves, B.B., Robinson, T.N.: Digital Trace Data Col- lection for Social Media Effects Research: APIs, Data Dona- tion, and (Screen) Tracking. Communication Methods and Mea- sures pp. 1-18 (Feb 2023), ISSN 1931-2458, 1931-2466, URL https://www.tandfonline.com/doi/full/10.1080/19312458.2023.2181319 · DOI · Paper page
  33. Power, M.: The audit society: Rituals of verification. British Journal of Educational Studies 47(1), 92-94 (1999)
  34. Radiya-Dixit, E., Neff, G.: A sociotechnical audit: Assessing police use of facial recognition. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, p. 1334-1346, FAccT '23, As- sociation for Computing Machinery, New York, NY, USA (2023), ISBN 9798400701924, URL https://doi.org/10.1145/3593013.3594084 · DOI
  35. Raji, I.D., Buolamwini, J.: Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Prod- ucts. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 429-435, ACM, Honolulu HI USA (Jan 2019), ISBN 978-1- 4503-6324-2, URL https://dl.acm.org/doi/10.1145/3306618.3314244 · DOI · Paper page
  36. Rakova, B., Dobbe, R.: Algorithms as Social-Ecological-Technological Sys- tems: an Environmental Justice Lens on Algorithmic Audits. In: 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 491-491 (Jun 2023), URL http://arxiv.org/abs/2305.05733, arXiv:2305.05733 [cs] · arXiv
  37. Sambasivan, N., Arnesen, E., Hutchinson, B., Prabhakaran, V.: Non-portability of Algorithmic Fairness in India (Dec 2020), URL http://arxiv.org/abs/2012.03659, arXiv:2012.03659 [cs] · arXiv
  38. Sartori, L., Theodorou, A.: A sociotechnical perspective for the future of ai: narratives, inequalities, and human control. Ethics and Information Technology 24(4), 315- 328 (2022), https://doi.org/10.1007/s10676-022-09624-3, URL https://doi.org/10.1007/s10676-022-09624-3 · DOI
  39. Selinger, E., Hartzog, W.: Facebook's emotional conta- gion study and the ethical problem of co-opted identity in mediated environments where users lack control. Re- search Ethics 12(1), 35-43 (Jan 2016), ISSN 1747-0161, URL https://journals.sagepub.com/doi/full/10.1177/1747016115579531, publisher: SAGE Publications Ltd · DOI
  40. Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., Virk, G.: So- ciotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. In: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, p. 723-741, AIES '23, Association for Comput- ing Machinery, New York, NY, USA (2023), ISBN 9798400702310, URL https://doi.org/10.1145/3600211.3604673 · DOI · Paper page
  41. Sikka, P.: Financial crisis and the silence of the auditors. Accounting, Or- ganizations and Society (2009), https://doi.org/10.1016/j.aos.2009.01.004 · DOI
  42. Spindler, G.: Die vorschläge der eu-kommission zu einer neuen produk- thaftung und zur haftung von herstellern und betreibern künstlicher intelligenz. Computer und Recht 38(11), 689-704 (2022), URL https://doi.org/10.9785/cr-2022-381106 · DOI
  43. Stahl, B.C., Antoniou, J., Bhalla, N., et al.: A systematic review of artificial intelligence impact assessments. Artificial Intelligence Review (2023)
  44. Sweeney, L.: Discrimination in Online Ad Delivery. SSRN Electronic Journal (2013), ISSN 1556-5068, URL http://www.ssrn.com/abstract=2208240
  45. Thelisson, E., Verma, H.: Conformity assessment under the EU AI act general approach. AI and Ethics 4(1), 113-121 (Feb 2024), ISSN 2730- 5961, URL https://doi.org/10.1007/s43681-023-00402-5 · DOI
  46. Thomas, L.D.W., Autio, E.: Innovation Ecosystems (Oct 2019), URL https://papers.ssrn.com/abstract=3476925
  47. Trezza, D.: To scrape or not to scrape, this is dilemma. The post-API scenario and implications on digital research. Frontiers in Sociology 8 (2023), ISSN 2297-7775, URL https://www.frontiersin.org/articles/10.3389/fsoc.2023.1145038 · DOI
  48. Vakkuri, V., Kemell, K.K., Jantunen, M., Abrahamsson, P.: "This is Just a Prototype": How Ethics Are Ignored in Software Startup-Like Environ- ments. In: Stray, V., Hoda, R., Paasivaara, M., Kruchten, P. (eds.) Agile Processes in Software Engineering and Extreme Programming, pp. 195- 210, Lecture Notes in Business Information Processing, Springer Interna- tional Publishing, Cham (2020), ISBN 978-3-030-49392-9
  49. Vakkuri, V., Kemell, K.K., Kultanen, J., Siponen, M., Abrahamsson, P.: Ethically Aligned Design of Autonomous Systems: Industry Viewpoint and an Empirical Study. EJBO Electronic Journal of Business Ethics and Or- ganization Studies 27(1) (2022)
  50. van de Waerdt, P.J.: Information asymmetries: recognizing the limits of the gdpr on the data-driven market. Computer Law & Security Review 38, 105436 (2020), ISSN 0267-3649, URL https://www.sciencedirect.com/science/article/pii/S0267364920300418
  51. Veale, M.: Governing Machine Learning that Matters. Ph.D. thesis, Uni- versity College London (UCL) (Aug 2019)
  52. Wehde: Datenzugang über art. 31 abs. 2 dsa-e. MMR pp. 827-830 (2022)
  53. Wieringa, M.: What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In: Proceed- ings of the 2020 Conference on Fairness, Accountability, and Trans- parency, pp. 1-18, FAT* '20, Association for Computing Machin- ery, New York, NY, USA (Jan 2020), ISBN 978-1-4503-6936-7, URL https://dl.acm.org/doi/10.1145/3351095.3372833 · DOI · Paper page
  54. Wilson, C., Ghosh, A., Jiang, S., Mislove, A., Baker, L., Szary, J., Trindel, K., Polli, F.: Building and auditing fair algorithms: A case study in candi- date screening. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, p. 666-677, FAccT '21, Association for Computing Machinery, New York, NY, USA (2021), ISBN 9781450383097, URL https://doi.org/10.1145/3442188.3445928 · DOI · Paper page
  55. Wilson, C., Ghosh, A., Jiang, S., Mislove, A., Baker, L., Szary, J., Trindel, K., Polli, F.: Building and Auditing Fair Algorithms: A Case Study in Candidate Screening. In: Proceedings of the 2021 ACM Con- ference on Fairness, Accountability, and Transparency, pp. 666-677, ACM, Virtual Event Canada (Mar 2021), ISBN 978-1-4503-8309-7, URL https://dl.acm.org/doi/10.1145/3442188.3445928 · DOI · Paper page
  56. Young, M., Katell, M., Krafft, P.: Confronting power and corporate cap- ture at the facct conference. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, p. 1375-1386, FAccT '22, Association for Computing Machinery, New York, NY, USA (2022), ISBN 9781450393522, URL https://doi.org/10.1145/3531146.3533194 · DOI

Cited by

Sorted by citations

Authors
Hartmann, D., de Pereira, J. R. L., Streitbörger, C., & Berendt, B.
Affiliations
Weizenbaum Institute (DE)
Corresponding author
Hartmann, David
ORCID
Venue
AI and Ethics
Publisher
Springer Science and Business Media LLC
Published
2024-11-28
Volume
5
Issue
4
Pages
3617-3638
ISSN
  • 2730-5953 (ISSN-L)
  • 2730-5961
DOI
10.1007/s43681-024-00595-3
OpenAlex
W4404824525
Cited by
58
References
11
Citations by year
2026: 31 · 2025: 22 · 2024: 5
FWCI
82.87
Type
journal article
Language
en
Open access
Licence
Locations
  • AI and Ethics (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Weizenbaum-Institut e.V.
Topics
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
EU AI Act, Digital Services Act, third-party auditing, legislative gaps, civil society oversight, research data access, high-risk AI systems, delegated acts, AI regulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review