CoChat

Guardrails for Human-AI Ecologies: Norm-Based Coordination and Design for Predictability

Grisold, Thomas, Berente, Nicholas, Seidel, Stefan · 2025 · MIS Quarterly
Cited by 24

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.

Abstract

Human-AI ecologies involve human and AI-based agents that coordinate their interactions in part by following social norms. Social norms, therefore, are important for establishing the guardrails that ensure desirable interactions in a way that is consistent with essential values, such as human safety. Managing human-AI ecologies requires specifying norms to enable coordination in known situations but also allowing for the emergence of norms to enable coordination in unspecified, unstructured situations. We integrate predictive processing theory and social norm theory to explain how existing norms are enacted and reinforced based on agents’ predictive models and how new norms emerge as agents update their predictive models in response to prediction errors in uncertain coordination scenarios. Rooted in this perspective, we develop a design theory that emphasizes design for predictability and propose a set of design principles for managers and developers to encode norms to evolve in human-AI ecologies, monitor outcomes, and intervene when necessary.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.25300/misq/2025/18058

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 23 references

Research works printed but not linked: 67

  1. Abbink, 2017, Peer punishment promotes enforcement of bad social norms, Nature Communications, 8, 10.1038/s41467-017-00731-0 · DOI
  2. Abrutyn, 2015, The decline in shared collective conscience as found in the shifting norms and values of etiquette manuals, Journal for the Theory of Social Behaviour, 45, 352, 10.1111/jtsb.12071 · DOI
  3. Argote, 1982, Input uncertainty and organizational coordination in hospital emergency units, Administrative Science Quarterly, 27, 420, 10.2307/2392320 · DOI · Paper page
  4. Baldwin, 2011, Understanding regulation: theory, strategy, and practice, 10.1093/acprof:osobl/9780199576081.001.0001 · DOI
  5. Bennett Moses, 2016, Regulating in the face of sociotechnical change, 573, 10.1093/oxfordhb/9780199680832.013.49 · DOI
  6. Berente, 2011, Ethical implications of social stratification in information systems research, Information Systems Journal, 21, 357, 10.1111/j.1365-2575.2010.00353.x · DOI
  7. Berente, 2024, Test-driven ethics for machine learning, Communications of the ACM, 67, 45, 10.1145/3633455 · DOI
  8. Browne, 2022, Extended, distributed, and predictive: Sketches of a generative theory of interaction for HCI, 17, 10.1007/978-3-031-19679-9_3 · DOI
  9. Burgemeestre, 2009, Rule-based versus principle-based regulatory compliance, 37, 10.3233/978-1-60750-082-7-37 · DOI
  10. Butler, 2023, The regulation of and through information technology: Towards a conceptual ontology for IS research, Journal of Information Technology, 38, 86, 10.1177/02683962231181147 · DOI
  11. Carroll, 1997, Human-computer interaction: psychology as a science of design, Annual Review of Psychology, 48, 61, 10.1006/ijhc.1996.0101 · DOI
  12. Carter, 2011, Principles-based regulation and emerging technology, 157, 10.1007/978-94-007-1356-7_10 · doi:10.1007/978-94-007-1356-7_10
  13. Chopra, 2018, Handook of normative multiagent systems
  14. Chung, 2016, Social norms: A review, Review of Communication Research, 4, 1, 10.12840/issn.2255-4165.2016.04.01.008 · DOI · Paper page
  15. Churchman, 1971, The design of inquiring systems: Basic concepts of systems and organization, Basic Books
  16. Claggett, 2018, Unpacking the structure of coordination mechanisms and the role of relational coordination in an era of digitally mediated work processes, Academy of Management Review, 43, 704, 10.5465/amr.2016.0325 · DOI · Paper page
  17. Crowston, 1997, A coordination theory approach to organizational process design, Organization Science, 8, 157, 10.1287/orsc.8.2.157 · DOI · Paper page
  18. de Freitas, 2017, Multiagent system for image mining, Multi-agent Systems. IntechOpen, 10.5772/intechopen.69353 · DOI
  19. Diekmann, 2022, Emergence of and compliance with new social norms: The example of the COVID crisis in Germany, Rationality and Society, 34, 129, 10.1177/10434631221092 · doi:10.1177/10434631221092
  20. Feldman, 1984, The development and enforcement of group norms, Academy of Management Review, 9, 47, 10.2307/258231 · doi:10.2307/258231 · Paper page
  21. Frantz, 2018, Modeling norm dynamics in multiagent systems, 491
  22. Frauenberger, 2019, Entanglement HCI the next wave?, ACM Transactions on Computer-Human Interaction, 27, 1, 10.1145/336499 · doi:10.1145/336499
  23. Gasser, 2024, Guardrails: Guiding human decisions in the age of AI
  24. Gibson, 1979, The ecological approach to visual perception
  25. Gittell, 2002, Coordinating mechanisms in care provider groups: Relational coordination as a mediator and input uncertainty as a moderator of performance effects, Management Science, 48, 1408, 10.1287/mnsc.48.11.1408.268 · DOI · Paper page
  26. Gregor, 2020, Research perspectives: the anatomy of a design principle, Journal of the Association for Information Systems, 21, 1622, 10.17705/1jais.00649 · DOI · Paper page
  27. Harrison, 2011, Making epistemological trouble: Third-paradigm HCI as successor science, Interacting with Computers, 23, 385, 10.1016/j.intcom.2011.03.005 · DOI · Paper page
  28. Hawkins, 2019, The emergence of social norms and conventions, Trends in Cognitive Sciences, 23, 158, 10.1016/j.tics.2018.11.003 · DOI
  29. Iacono, 2021, COVID-19 vulnerability and perceived norm violations predict loss of social trust: A pre-post study, Social Science & Medicine, 291, 10.1016/j.socscimed.2021.114513 · DOI
  30. Inam, 2018, Risk assessment for human-robot collaboration in an automated warehouse scenario, 10.1109/ETFA.2018.8502466 · DOI
  31. Jennings, 2014, Human-agent collectives, Communications of the ACM, 57, 80, 10.1145/2629559 · DOI · Paper page
  32. Kane, 2021, Avoiding an oppressive future of machine learning: A design theory for emancipatory assistants, MIS Quarterly, 45, 371, 10.25300/MISQ/2021/1578 · DOI
  33. Kelly, 2019, The brain, self and society: a social-neuroscience model of predictive processing, Social Neuroscience, 14, 266, 10.1080/17470919.2018.1471003 · DOI
  34. Kissinger, 2021, The age of AI and our human future
  35. Kiverstein, 2012, Dealing with context through action-oriented predictive processing, Frontiers in Psychology, 3, 10.3389/fpsyg.2012.00421 · DOI
  36. Knight, 2025, Sensing the room: the role of atmosphere in collective sensemaking, Academy of Management Journal, 68, 326, 10.5465/amj.2021.1389 · DOI
  37. Koops, 2006, Should ICT regulation be technology-neutral?, 77
  38. Köster, 2020, Making sense of the world: infant learning from a predictive processing perspective, Perspectives on Psychological Science, 15, 562, 10.1177/1745691619895071 · DOI · Paper page
  39. Kuutti, 2014, The turn to practice in HCI: towards a research agenda, 10.1145/2556288.2557111 · DOI · Paper page
  40. Lanillos, 2018, Adaptive robot body learning and estimation through predictive coding, 10.1109/IROS.2018.8593684 · DOI · Paper page
  41. Lewis, 1969, Convention
  42. Linson, 2018, The active inference approach to ecological perception: General information dynamics for natural and artificial embodied cognition, Frontiers in Robotics and AI, 5, 10.3389/frobt.2018.00021 · DOI · Paper page
  43. Lobo, 2018, The history and philosophy of ecological psychology, Frontiers in Psychology, 9, 10.3389/fpsyg.2018.02228 · DOI · Paper page
  44. Lupyan, 2015, Words and the world: Predictive coding and the language-perception-cognition interface, Current Directions in Psychological Science, 24, 279, 10.1177/0963721415570732 · DOI · Paper page
  45. Lyytinen, 2021, Metahuman systems=humans+machines that learn, Journal of Information Technology, 36, 427, 10.1177/02683962209159 · doi:10.1177/02683962209159
  46. Mandel, 2009, Regulating emerging technologies, Law, Innovation and Technology, 1, 75, 10.1080/17579961.2009.11428365 · DOI · Paper page
  47. Marmor, 2009, Social conventions—From language to law
  48. Przepiorka, 2022, How norms emerge from conventions (and change), Socius, 8, 1, 10.1177/23780231221124556 · DOI
  49. Raisch, 2025, Combining human and artificial intelligence: Hybrid problem-solving in organizations, Academy of Management Review, 50, 441, 10.5465/amr.2021.0421 · DOI · Paper page
  50. Reichardt, 2020, Novelty manipulations, memory performance, and predictive coding: The role of unexpectedness, Frontiers in Human Neuroscience, 14, 10.3389/fnhum.2020.00152 · DOI
  51. Russell, 2020, Artificial intelligence: A modern approach
  52. Safadi, 2024, The effect of bots on human interaction in online communities, MIS Quarterly, 48, 1279, 10.25300/MISQ/2023/17901 · DOI
  53. Savarimuthu, 2011, Norm creation, spreading and emergence: A survey of simulation models of norms in multi-agent systems, Multiagent and Grid Systems, 7, 21, 10.5555/3635637.3663046 · doi:10.5555/3635637.3663046
  54. Schölkopf, 2022, Causality for machine learning, 765, 10.1145/3501714 · DOI
  55. Seidel, 2018, Autonomous tools and design: A triple-loop approach to human-machine learning, Communications of the ACM, 62, 50, 10.1145/3210753 · DOI · Paper page
  56. Seidel, 2025, Regulating emerging technologies: Prospective sensemaking through abstraction and elaboration, MIS Quarterly, 49, 179, 10.25300/MISQ/2024/18039 · DOI
  57. Stelmaszak, 2024, When algorithms delegate to humans: Exploring human-algorithm interaction at Uber, MIS Quarterly, 49, 305, 10.25300/MISQ/2024/17911 · DOI
  58. Tamir, 2018, Modeling the predictive social mind, Trends in Cognitive Sciences, 22, 201, 10.1016/j.tics.2017.12.005 · DOI · Paper page
  59. Taniguchi, 2023, World models and predictive coding for cognitive and developmental robotics: Frontiers and challenges, Advanced Robotics, 37, 780, 10.1080/01691864.2023.2225232 · DOI · Paper page
  60. Te’eni, 2023, Reciprocal human-machine learning: A theory and an instantiation for the case of message classification, Management Science, Articles in Advance, 10.1287/mnsc.2022.03518 · DOI
  61. Ullman-Margalit, 1977, Coordination norms and social choice, Erkenntnis, 143, 10.1007/BF00169850 · DOI
  62. Ullmann-Margalit, 1977, The emergence of norms
  63. Weiss, 1999, Multiagent systems: a modern approach to distributed artificial intelligence
  64. Wheeler, 2020, Ideology and predictive processing: coordination, bias, and polarization in socially constrained error minimization, Current Opinion in Behavioral Sciences, 34, 192, 10.1016/j.cobeha.2020.05.002 · DOI
  65. Wurman, 2008, Coordinating hundreds of cooperative, autonomous vehicles in warehouses, AI Magazine, 29, 9, 10.1609/aimag.v29i1.2082 · DOI
  66. Yakura, H., Lopez-Lopez, E., Brinkmann, L., Serna, I., Gupta, P., & Rahwan, I. (2024). Empirical evidence of large language model’s influence on human spoken communication (Working paper). arXiv. https://arxiv.org/abs/2409.01754 · arXiv
  67. Young, 2021, Organizing workers and machine learning tools for a less oppressive workplace, International Journal of Information Management, 59, 10.1016/j.ijinfomgt.2021.102353 · DOI

Cited by

See all 24

Metadata

Authors
Grisold, T., Berente, N., & Seidel, S.
Affiliations
  • WU Vienna University of Economics and Business, Vienna, AUSTRIA, and University of St. Gallen, St. Gallen, Switzerland
  • University of Notre Dame South Bend, IN, U.S.A.
  • University of Cologne Cologne, Germany
Venue
MIS Quarterly
Publisher
MIS Quarterly
Published
2025-12-01
Volume
49
Issue
4
Pages
1239-1266
ISSN
  • 0276-7783
  • 2162-9730
Cited by
24
References
23
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Guardrails for Human-AI Ecologies: Norm-Based Coordination and Design for Predictability. https://app.cochat.ai/papers/10.25300/misq/2025/18058
The paper
Grisold, T., Berente, N., & Seidel, S. (2025). Guardrails for Human-AI Ecologies: Norm-Based Coordination and Design for Predictability. MIS Quarterly. https://doi.org/10.25300/misq/2025/18058

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.