CoChat
Sign Up
Cited by 95Ethics and Social Impacts of AIDigitalization, Law, and RegulationExplainable Artificial Intelligence (XAI)

How to regulate algorithmic decision-making: A framework of regulatory requirements for different applications

Open access
Krafft, Tobias D., Zweig, Katharina A., König, Pascal D.·2022·Regulation & Governance
DOI 10.1111/rego.12369
Open this paper in CoChat — free accountOpen PDF

Abstract

Algorithmic decision‐making (ADM) systems have come to support, pre‐empt or substitute for human decisions in manifold areas, with potentially significant impacts on individuals' lives. Achieving transparency and accountability has been formulated as a general goal regarding the use of these systems. However, concrete applications differ widely in the degree of risk and the accountability problems they entail for data subjects. The present paper addresses this variation and presents a framework that differentiates regulatory requirements for a range of ADM system uses. It draws on agency theory to conceptualize accountability challenges from the point of view of data subjects with the purpose to systematize instruments for safeguarding algorithmic accountability. The paper furthermore shows how such instruments can be matched to applications of ADM based on a risk matrix. The resulting comprehensive framework can guide the evaluation of ADM systems and the choice of suitable regulatory provisions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/rego.12369

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The consequences of regulatory gaps including bias in training data sets, unethical labour practices, and the predatory role of data brokers are widely documented in the literature.

    “The consequences of regulatory gaps including bias in training data sets, unethical labour practices, and the predatory role of data brokers are widely documented in the literature (Krafft et al., 2022;Noble, 2018;Smuha, 2021).”

    From Representation of Libraries in Artificial Intelligence Regulations and Implications for Ethics and Practice (2022) · cites this paper as Krafft et al., 2022

    Section 2.1 says discriminatory bias can be learned from training data, and the discussion of GDPR limitations describes gaps in regulating algorithmic decision-making. The cited paper does not document unethical labour practices or the predatory role of data brokers, so it supports only part of the sentence. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • One way to address the balance between legitimacy concerns and administrative burden and flexibility is to calibrate institutional arrangements based on the degree of risk of an algorithm.

    “One way to address this balance is to calibrate institutional arrangement based on the degree of risk of an algorithm (Krafft, Zweig, and König 2022).”

    From Legitimacy of Algorithmic Decision-Making: Six Threats and the Need for a Calibrated Institutional Response (2022) · cites this paper as Krafft, Zweig, and König 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 54

  1. Abbott KW, Levi-faur D, Snidal D (2017) Theorizing Regulatory Intermediaries: The RIT Model. The Annals of the American Academy of Political and Social Science 670, 14-35. https://doi.org/10.1177/0002716216688272. · DOI
  2. Black J (2010) Risk-Based Regulation: Choices, Practices and Lessons Being Learnt. In: OECD (ed) Risk and Regulatory Policy: Improving the Governance of Risk. OECD Reviews of Regulatory Reform, pp. 185-236. OECD, Paris. https://doi.org/10. 1787/9789264082939-en. · DOI
  3. Black J, Baldwin R (2012) When Risk-Based Regulation Aims Low: Approaches and Challenges: Aiming Low: Approaches and Challenges. Regulation & Governance 6, 2-22. https://doi.org/10.1111/j.1748-5991.2011.01124.x. · DOI
  4. Brauneis R, Goodman EP (2018) Algorithmic Transparency for the Smart City. The Yale Journal of Law and Technology 20, 103-175. https://doi.org/10.2139/ssrn.3012499. · DOI
  5. Brkan M (2019) Do Algorithms Rule the World? Algorithmic Decision-Making and Data Protection in the Framework of the GDPR and beyond. International Journal of Law and Information Technology 27, 91-121. https://doi.org/10.1093/ijlit/ eay017. · DOI
  6. Bryson JJ, Theodorou A (2019) How Society Can Maintain Human-Centric Artificial Intelligence. In: Toivonen M, Saari E (eds) Human-Centered Digitalization and Services, Vol. 19, pp. 305-323. Springer Singapore, Singapore. https://doi.org/ 10.1007/978-981-13-7725-9_16. · DOI · Paper page
  7. Bygrave LA (2019) Minding the machine v2.0: The EU General Data Protection Regulation and Automated Decision Making. In: Algorithmic Regulation (ed) Karen Yeung and Martin Lodge, pp. 248-262. Oxford University Press, Oxford.
  8. Diakopoulos N (2014) Algorithmic Accountability Reporting: On the Investigation of Black Boxes. Columbia University. https:// doi.org/10.7916/d8zk5tw2. · DOI
  9. van Drunen MZ, Helberger N, Bastian M (2019) Know your Algorithm: What Media Organizations Need to Explain to Their Users about News Personalization. International Data Privacy Law 9, 1-16. https://doi.org/10.1093/idpl/ipz011. · DOI
  10. Ebrahim A (2003) Making Sense of Accountability: Conceptual Perspectives for Northern and Southern Nonprofits. Nonprofit Management and Leadership 14, 191-212. https://doi.org/10.1002/nml. · DOI
  11. Eyert F, Irgmaier F, Ulbricht L (2022) Extending the framework of algorithmic regulation. The Uber case. Regulation & Gover- nance 16(1), 23-44.
  12. Gellert R (2022) Comparing Definitions of Data and Information in Data Protection Law and Machine Learning: A Useful Way Forward to Meaningfully Regulate Algorithms? Regulation & Governance 16(1), 156-176.
  13. Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D (2018) A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys 51, 1-42. https://doi.org/10.1145/3236009. · DOI · Paper page
  14. Heremans T (2012) Professional Services in the EU Internal Market: Quality Regulation and Self-Regulation. Hart, Oxford.
  15. Hildebrandt M (2008) Defining Profiling: A New Type of Knowledge? In: Hildebrandt M, Gutwirth S (eds) Profiling the European Citizen, pp. 17-45. Springer Netherlands, Dordrecht. https://doi.org/10.1007/978-1-4020-6914-7_2. · DOI
  16. Hildebrandt M (2016) Law as Information in the Era of Data-Driven Agency: Law as Information. Modern Law Review 79, 1-30. https://doi.org/10.1111/1468-2230.12165. · DOI
  17. Hill CWL, Jones TM (1992) Stakeholder-Agency Theory. Journal of Management Studies 29, 131-154. https://doi.org/10.1111/ j.1467-6486.1992.tb00657.x. · DOI
  18. Hirschmann AO (1970) Exit, Voice and Loyalty: Responses to Decline in Firms, Organizations and States. Harvard University Press, Cambridge.
  19. Hölmstrom B (1979) Moral Hazard and Observability. The Bell Journal of Economics 10, 74-91.
  20. Hölmstrom B, Milgrom P (1987) Aggregation and Linearity in the Provision of Intertemporal Incentives. Econometrica 55, 303-328. https://doi.org/10.2307/1913238. · DOI
  21. Jensen MC, Meckling WH (1976) Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure. Journal of Financial Economics 3, 305-360. https://doi.org/10.1016/0304-405X(76)90026-X. · DOI · Paper page
  22. Just N, Latzer M (2017) Governance by Algorithms: Reality Construction by Algorithmic Selection on the Internet. Media, Culture & Society 39, 238-258. https://doi.org/10.1177/0163443716643157. · DOI
  23. Kerwer D (2005) Holding Global Regulators Accountable: The Case of Credit Rating Agencies. Governance 18, 453-475. https://doi.org/10.1111/j.1468-0491.2005.00284.x. · DOI
  24. Hildebrandt M, de Vries K (eds) Privacy, Due Process and the Computational Turn: The Philosophy of Law Meets the Phi- losophy of Technology, pp. 196-220.
  25. Milton Park, Routledge.
  26. Kroll JA, Huey J, Barocas S et al. (2017) Accountable Algorithms. University of Pennsylvania Law Review 165, 633-705.
  27. Lane J-E (2007) Comparative Politics: The Principal-Agent Perspective. Routledge, Milton Park. https://doi.org/10.4324/ 9780203935545. · DOI
  28. Lanzing M (2018) "Strongly Recommended" Revisiting Decisional Privacy to Judge Hypernudging in Self-Tracking Technolo- gies. Philosophy & Technology 32, 549-568. https://doi.org/10.1007/s13347-018-0316-4. · DOI
  29. Lepri B, Oliver N, Letouzé E, Pentland A, Vinck P (2018) Fair, Transparent, and Accountable Algorithmic Decision-Making Processes: The Premise, the Proposed Solutions, and the Open Challenges. Philosophy & Technology 31, 611-627. https:// doi.org/10.1007/s13347-017-0279-x. · DOI · Paper page
  30. Lodge M (2004) Accountability and Transparency in Regulation: Critiques, Doctrines and Instruments. In: Jordana J, Levi- Faur D (eds) The Politics of Regulation, pp. 124-144. Cheltenham: Edward Elgar Publishing.
  31. Lupia A (2003) Delegation and Its Perils. In: Strøm K, Bergman T, Müller WC (eds) Delegation and Accountability in Parlia- mentary Democracies, pp. 33-54. Oxford University Press, Oxford; New York.
  32. MacNeil I (2010) Risk Control Strategies: An Assessment in the Context of the Credit Crisis. In: MacNeil I, O'Brien J (eds) The Future of Financial Regulation, pp. 141-160.
  33. Miller GJ (2005) The Political Evolution of Principal Agent Models. Annual Review of Political Science 8, 203-225. https://doi. org/10.1146/annurev.polisci.8.082103.104840. · DOI
  34. Nalebuff BJ, Stiglitz JE (1983) Prizes and Incentives: Towards a General Theory of Compensation and Competition. The Bell Journal of Economics 14, 21. https://doi.org/10.2307/3003535. · DOI
  35. Nemitz P (2018) Constitutional Democracy and Technology in the Age of Artificial Intelligence. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376, 1-14. https://doi.org/10.1098/rsta.2018.0089. · DOI · Paper page
  36. Osterloh M, Frey BS (2002) Does Pay for Performance Really Motivate Employees. In: Neely AD (ed) Business Performance Measurement: Theory and Practice, pp. 107-122. Cambridge University Press, Cambridge.
  37. Pasquale F (2015) The Black Box Society: The Secret Algorithms that Control Money and Information. Harvard University Press, Cambridge, MA; London.
  38. Patil SV, Vieider F, Tetlock PE (2014) Process Versus Outcome Accountability. In: Patil SV, Vieider F, Tetlock PE (eds) The Oxford Handbook of Public Accountability, pp. 69-89. Oxford University Press, Oxford.
  39. Pratt JW, Zeckhauser R (1991) Principals and Agents: An Overview. In: Pratt JW, Zeckhauser R (eds) Principals and Agents: The Structure of Business. Research Colloquium, pp. 1-36. Harvard Business School Press, Boston, MA.
  40. Robinson H, Carrillo P, Anumba CJ, Patel M (2010) Governance and Knowledge Management for Public-Private Partnerships. Wiley-Blackwell, Chichester; Malden, MA.
  41. Rudin C (2019) Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead. Nature Machine Intelligence 1, 206-215. https://doi.org/10.1038/s42256-019-0048-x. · DOI · Paper page
  42. Saam NJ (2007) Asymmetry in Information Versus Asymmetry in Power: Implicit Assumptions of Agency Theory? The Jour- nal of Socio-Economics 36, 825-840. https://doi.org/10.1016/j.socec.2007.01.018. · DOI
  43. Sappington DEM (1991) Incentives in Principal-Agent Relationships. Journal of Economic Perspectives 5, 45-66. https://doi. org/10.1257/jep.5.2.45. · DOI
  44. Saurwein F, Just N, Latzer M (2015) Governance of Algorithms: Options and Limitations. Info 17, 35-49. https://doi.org/10. 1108/info-05-2015-0025. · DOI
  45. Schiffman LG, Kanuk LL, Hansen H (2012) Consumer Behaviour: A European Outlook, 2nd edn. Pearson Financial Times/Prentice Hall, Harlow; New York.
  46. Sherman R (2011) The Regulation of Monopoly. Cambridge University Press, Cambridge.
  47. Sokol K, Flach P (2020) Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches. Pro- ceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 56-67. ACM, Barcelona. https://doi. org/10.1145/3351095.3372870. · DOI
  48. Tene O, Polonetsky J (2017) Taming the Golem: Challenges of Ethical Algorithmic Decision-Making. North Carolina Journal of Law & Technology 19, 125-173.
  49. Ulbricht L, Yeung K (2022) Algorithmic Regulation: A Maturing Concept for Investigating Regulation of and Through Algo- rithms. Regulation & Governance 16(1), 3-22.
  50. Warren ME (2014) Accountability and Democracy. In: Bovens M, Goodin RE, Schillemans T (eds) The Oxford Handbook of Public Accountability, pp. 39-54. Oxford University Press, Oxford.
  51. Watt J, Borhani R, Katsaggelos AK (2020) Machine Learning Refined: Foundations, Algorithms, and Applications, Second edn. Cambridge University Press, New York.
  52. Weingast BR, Moran MJ (1983) Bureaucratic Discretion or Congressional Control? Regulatory Policymaking by the Federal Trade Commission. Journal of Political Economy 91, 765-800. https://doi.org/10.1086/261181. · DOI
  53. Weiss MD (1995) Information Issues for Principals and Agents in the "Market" for Food Safety and Nutrition. In: Caswell JA (ed) Valuing Food Safety and Nutrition, pp. 69-82. Routledge, Milton Park.
  54. Wieringa M (2020) What to Account for when Accounting for Algorithms: A Systematic Literature Review on Algorithmic Accountability. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 1-18. ACM, Barcelona. https://doi.org/10.1145/3351095.3372833. · DOI · Paper page

Cited by

Sorted by citations

Authors
Krafft, T. D., Zweig, K. A., & König, P. D.
Affiliations
University of Kaiserslautern (DE)
Corresponding author
Krafft, Tobias D.
ORCID
Venue
Regulation & Governance
Publisher
Wiley
Published
2020-10-20
Volume
16
Issue
1
Pages
119-136
ISSN
  • 1748-5983 (ISSN-L)
  • 1748-5991
DOI
10.1111/rego.12369
OpenAlex
W3094585180
Cited by
95
References
13
Citations by year
2026: 21 · 2025: 15 · 2024: 19 · 2023: 13 · 2022: 18 · 2021: 8 · 2020: 1
FWCI
13.61
Type
journal article
Language
en
Open access
Licence
Locations
  • Regulation & Governance (publishedVersion, cc-by)
  • Econstor (Econstor) (submittedVersion, cc-by)
  • Publication Server of Kaiserslautern University of Technology (Kaiserslautern University of Technology) (submittedVersion)
  • VU Research Portal (submittedVersion)
Funders
Volkswagen Foundation
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic decision-making, algorithmic accountability, algorithmic transparency, regulatory requirements, risk-based regulation, risk matrix, agency theory, data subjects, automated decision-making, regulatory framework

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review