CoChat
Sign Up
Cited by 4Merger and Competition AnalysisDigital Platforms and EconomicsAuction Theory and Applications

Learning to Mitigate AI Collusion on Economic Platforms

Open access
Brero, Gianluca, Mibuari, Eric, Lepore, Nicolas, Parkes, David·2022·Advances in Neural Information Processing Systems 35
DOI 10.52202/068431-2746
Open this paper in CoChat — free accountOpen PDF

Abstract

Algorithmic pricing on online e-commerce platforms raises the concern of tacit collusion, where reinforcement learning algorithms learn to set collusive prices in a decentralized manner and through nothing more than profit feedback. This raises the question as to whether collusive pricing can be prevented through the design of suitable "buy boxes," i.e., through the design of the rules that govern the elements of e-commerce sites that promote particular products and prices to consumers. In this paper, we demonstrate that reinforcement learning (RL) can also be used by platforms to learn buy box rules that are effective in preventing collusion by RL sellers. For this, we adopt the methodology of Stackelberg POMDPs, and demonstrate success in learning robust rules that continue to provide high consumer welfare together with sellers employing different behavior models or having out-of-distribution costs for goods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/068431-2746

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Brero et al. (2022) introduce an ML-based scheme to avoid collusion in buy box placement.

    “Exceptions include work by Brero et al. (2022), who introduce an ML-based scheme to avoid collusion in buy box placement”

    From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Brero et al., 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 41

  1. Ibrahim Abada and Xavier Lambin. 2020. Artificial Intelligence: Can Seemingly Collusive Outcomes Be Avoided? Available at SSRN 3559308 (2020).
  2. Simon P Anderson and Andre De Palma. 1992. The logit as a model of product differentiation. Oxford Economic Papers 44, 1 (1992), 51-67.
  3. Enrique Areyan Viqueira, Cyrus Cousins, Yasser Mohammad, and Amy Greenwald. 2019. Empirical Mechanism Design: Designing Mechanisms from Data. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence (Proceedings of Machine Learning Research, Vol. 115) . 1094-1104.
  4. Craig Boutilier. 1996. Planning, Learning and Coordination in Multiagent Decision Processes. In Proceedings of the 6th Conference on Theoretical Aspects of Rationality and Knowledge . 195-210.
  5. Gianluca Brero, Darshan Chakrabarti, Alon Eden, Matthias Gerstgrasser, Vincent Li, and David Parkes. 2021a. Learning Stackelberg Equilibria in Sequential Price Mechanisms. In ICML Workshop for Reinforcement Learning Theory .
  6. Gianluca Brero, Alon Eden, Matthias Gerstgrasser, David C. Parkes, and Duncan Rheingans-Yoo. 2021b. Reinforcement Learning of Sequential Price Mechanisms. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI . 5219-5227.
  7. Erik Brinkman and Michael P. Wellman. 2017. Empirical Mechanism Design for Optimizing Clearing Interval in Frequent Call Markets. In Proceedings of the 2017 ACM Conference on Economics and Computation . 205-221.
  8. Benedikt B¨ unz, Benjamin Lubin, and Sven Seuken. 2018. Designing Core-selecting Payment Rules: A Computational Search Approach. In Proceedings of the 2018 ACM Conference on Economics and Computation . 109.
  9. Emilio Calvano, Giacomo Calzolari, Vincenzo Denicol` o, Joseph E Harrington, and Sergio Pastorello. 2020b. Protecting consumers from collusive prices due to AI. Science 370, 6520 (2020), 1040-1042.
  10. Emilio Calvano, Giacomo Calzolari, Vincenzo Denicolo, and Sergio Pastorello. 2020a. Artificial intelligence, algorithmic pricing, and collusion. American Economic Review 110, 10 (2020), 3267-97.
  11. Michael J. Curry, Ping-Yeh Chiang, Tom Goldstein, and John Dickerson. 2020. Certifying Strategyproof Auction Networks. In Proc. 33rd Annual Conference on Neural Information Processing Systems .
  12. Michael J. Curry, Uro Lyi, Tom Goldstein, and John Dickerson. 2022. Learning Revenue-Maximizing Auctions With Differentiable Matching. In Proc. 25th International Conference on Artificial Intelligence and Statistics . Forthcoming.
  13. Paul Duetting, Zhe Feng, Harikrishna Narasimhan, David C. Parkes, and Sai Srivatsa Ravindranath. 2019. Optimal Auctions through Deep Learning. In Proceedings of the 36th International Conference on Machine Learning . 1706-1715.
  14. Tanner Fiez, Benjamin Chasnov, and Lillian J. Ratliff. 2020. Implicit Learning Dynamics in Stackelberg Games: Equilibria Characterization, Convergence Analysis, and Empirical Study. In Proceedings of the 37th International Conference on Machine Learning . 3133-3144.
  15. Drew Fudenberg and Jean Tirole. 1991. Game theory . MIT press.
  16. Scott Fujimoto, David Meger, Doina Precup, Ofir Nachum, and Shixiang Shane Gu. 2022. Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error. arXiv preprint arXiv:2201.12417 (2022). · arXiv
  17. Ivo Grondman, Lucian Busoniu, Gabriel AD Lopes, and Robert Babuska. 2012. A survey of actor-critic reinforcement learning: Standard and natural policy gradients. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 6 (2012), 1291-1307.
  18. Chi Jin, Praneeth Netrapalli, and Michael I. Jordan. 2020. What is Local Optimality in NonconvexNonconcave Minimax Optimization?. In Proceedings of the 37th International Conference on Machine Learning , Vol. 119. 4880-4889.
  19. Patrick R. Jordan, L. Julian Schvartzman, and Michael P. Wellman. 2010. Strategy exploration in empirical games. In 9th International Conference on Autonomous Agents and Multiagent Systems . 1131-1138.
  20. Christopher Kiekintveld and Michael P. Wellman. 2008. Selecting strategies using empirical game models: an experimental analysis of meta-strategies. In 7th International Joint Conference on Autonomous Agents and Multiagent Systems . 1095-1101.
  21. Kevin Kuo, Anthony Ostuni, Elizabeth Horishny, Michael J. Curry, Samuel Dooley, Ping-Yeh Chiang, Tom Goldstein, and John P. Dickerson. 2020. ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning. CoRR abs/2010.06398 (2020). · arXiv
  22. Shihui Li, Yi Wu, Xinyue Cui, Honghua Dong, Fei Fang, and Stuart Russell. 2019. Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient. Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 2019), 4213-4220.
  23. Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in Neural Information Processing Systems (NeurIPS) 30 (2017).
  24. David Mguni, Joel Jennings, Emilio Sison, Sergio Valcarcel Macua, Sofia Ceppi, and Enrique Munoz de Cote. 2019. Coordinating the Crowd: Inducing Desirable Equilibria in Non-Cooperative Systems. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems . 386394.
  25. Rajesh K. Mishra, Deepanshu Vasal, and Sriram Vishwanath. 2020. Model-free Reinforcement Learning for Stochastic Stackelberg Security Games. In 59th IEEE Conference on Decision and Control . 348-353.
  26. Neehar Peri, Michael J. Curry, Samuel Dooley, and John P. Dickerson. 2021. PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning. In Proceedings of the 35th Conference on Neural Information Processing Systems .
  27. Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. 2021. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research 22, 268 (2021), 1-8.
  28. Jad Rahme, Samy Jelassi, Joan Bruna, and S. Matthew Weinberg. 2021b. A Permutation-Equivariant Neural Network Architecture For Auction Design. In Proc. Thirty-Fifth AAAI Conference on Artificial Intelligence . 5664-5672.
  29. Jad Rahme, Samy Jelassi, and S. Matthew Weinberg. 2021a. Auction Learning as a Two-Player Game. In Proc. 9th International Conference on Learning Representations, ICLR .
  30. Sailik Sengupta and Subbarao Kambhampati. 2020. Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense. arXiv:2007.10457 [cs] (July 2020). arXiv: 2007.10457. · arXiv
  31. Weiran Shen, Binghui Peng, Hanpeng Liu, Michael Zhang, Ruohan Qian, Yan Hong, Zhi Guo, Zongyao Ding, Pengjun Lu, and Pingzhong Tang. 2020. Reinforcement mechanism design: With applications to dynamic pricing in sponsored search auctions. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 34. 2236-2243.
  32. W. Shen, P. Tang, and S. Zuo. 2019. Automated Mechanism Design via Neural Networks. In Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems .
  33. Zhenyu Shi, Runsheng Yu, Xinrun Wang, Rundong Wang, Youzhi Zhang, Hanjiang Lai, and Bo An. 2020. Learning Expensive Coordination: An Event-Based Deep RL Approach. In 8th International Conference on Learning Representations .
  34. Tianmin Shu and Yuandong Tian. 2019. Mˆ3RL: Mind-aware Multi-agent Management Reinforcement Learning. In 7th International Conference on Learning Representations .
  35. Andrea Tacchetti, DJ Strouse, Marta Garnelo, Thore Graepel, and Yoram Bachrach. 2019. A Neural Architecture for Designing Truthful and Efficient Auctions. CoRR 1907.05181 (2019).
  36. Pingzhong Tang. 2017. Reinforcement mechanism design.. In Proc. 26th Int Joint Conf. on Art. Intell. (IJCAI) . 5146-5150.
  37. K. Tharakunnel and S. Bhattacharyya. 2007. Leader-follower semi-markov decision problems: theoretical framework and approximate solution. In IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning . 111-118.
  38. Yevgeniy Vorobeychik, Christopher Kiekintveld, and Michael P. Wellman. 2006. Empirical mechanism design: methods, with application to a supply-chain scenario. In Proceedings 7th ACM Conference on Electronic Commerce (EC-2006) . 306-315.
  39. Kai Wang, Lily Xu, Andrew Perrault, Michael K. Reiter, and Milind Tambe. 2022. Coordinating Followers to Reach Better Equilibria: End-to-End Gradient Descent for Stackelberg Games. In AAAI Conference on Artificial Intelligence .
  40. Michael P. Wellman. 2006. Methods for Empirical Game-Theoretic Analysis. In Proceedings, The TwentyFirst National Conference on Artificial Intelligence . 1552-1556.
  41. Lily Xu, Andrew Perrault, Fei Fang, Haipeng Chen, and Milind Tambe. 2021. Robust Reinforcement Learning Under Minimax Regret for Green Security. In Proc. 37th Conference on Uncertainty in Artifical Intelligence (UAI-21) .

Cited by

Sorted by citations

Authors
Brero, G., Mibuari, E., Lepore, N., & Parkes, D.
ORCID
Parkes, David
Venue
Advances in Neural Information Processing Systems 35
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2022
Pages
37892-37904
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/068431-2746
OpenAlex
W7133220180
Cited by
4
References
0
Citations by year
2025: 2 · 2023: 1 · 2022: 1
FWCI
3.38
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (191253)
  • Advanced Research Projects Agency
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Merger and Competition Analysis
  • Social Sciences › Business, Management and Accounting › Strategy and Management › Digital Platforms and Economics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Auction Theory and Applications
Keywords
machine learning, platform competition, competition law

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review