A Reinforcement Learning Model to Assess Market Power Under Auction-Based Energy Pricing
Abstract
Auctions serve as a primary pricing mechanism in various market segments of a deregulated power industry. In day-ahead (DA) energy markets, strategies such as uniform price, discriminatory, and second-price uniform auctions result in different price settlements and thus offer different levels of market power. In this paper, we present a nonzero sum stochastic game theoretic model and a reinforcement learning (RL)-based solution framework that allow assessment of market power in DA markets. Since there are no available methods to obtain exact analytical solutions of stochastic games, an RL-based approach is utilized, which offers a computationally viable tool to obtain approximate solutions. These solutions provide effective bidding strategies for the DA market participants. The market powers associated with the bidding strategies are calculated using well-known indexes like Herfindahl-Hirschmann index and Lerner index and two new indices, quantity modulated price index (QMPI) and revenue-based market power index (RMPI), which are developed in this paper. The proposed RL-based methodology is tested on a sample network
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RL has been applied to predict bids and market power in electricity markets, and to set dynamic prices in more general settings.
“For example, RL has been applied to predict bids [654] and market power [572] in electricity markets, and to set dynamic prices in more general settings [510].”
From Tackling climate change with machine learning (2023) · cites this paper as [572]
Cites
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Puterman, M. L. (1994) · Wiley series in probability and statistics · doi:10.1002/9780470316887 · Cited by 6,057 · see citing works
Supply Function Equilibria in Oligopoly under Uncertainty
Klemperer, P. D., & Meyer, M. (1989) · Econometrica · doi:10.2307/1913707 · Cited by 1,415 · see citing works
Competitive Markov Decision Processes
Filar, J. A., & Vrieze, K. (1996) · Springer eBooks · doi:10.1007/978-1-4612-4054-9 · Cited by 1,202 · see citing works
Stoft, S. E. (2002) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470545584 · Cited by 1,173 · see citing works
Design and Analysis of Experiments
Amidan, B. G., Dean, A., & Voss, D. T. (2000) · Technometrics · doi:10.2307/1271085 · Cited by 1,161 · see citing works
Nash Q-Learning for General-Sum Stochastic Games
Hu, J., Wellman, M. P., & Boutilier, C. (2003) · Cited by 909 · see citing works
Strategic gaming analysis for electric power systems: an MPEC approach
Hobbs, B. F., Metzler, C. B., & Pang, J. (2000) · IEEE Transactions on Power Systems · doi:10.1109/59.867153 · Cited by 834 · see citing works
Linear complementarity models of Nash-Cournot competition in bilateral and POOLCO power markets
Hobbs, B. (2001) · IEEE Transactions on Power Systems · doi:10.1109/59.918286 · Cited by 527 · see citing works
An Empirical Analysis of the Potential for Market Power in California’s Electricity Industry
Borenstein, S., & Bushnell, J. (1999) · Journal of Industrial Economics · doi:10.1111/1467-6451.00102 · Cited by 513 · see citing works
Linear Complementarity Models of Nash-Cournot Competition in Bilateral and POOLCO Power Markets
Hobbs, B. F. (2001) · IEEE Power Engineering Review · doi:10.1109/mper.2001.4311384 · Cited by 470 · see citing works
Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning
Gosavi, A. (2003) · Operations research, computer science. Interface series · doi:10.1007/978-1-4757-3766-0 · Cited by 369 · see citing works
Spatial Oligopolistic Electricity Models with Cournot Generators and Regulated Transmission Prices
Wei, J.-Y., & Smeers, Y. (1999) · Operations Research · doi:10.1287/opre.47.1.102 · Cited by 298 · see citing works
Solving Semi-Markov Decision Problems Using Average Reward Reinforcement Learning
Das, T. K., Gosavi, A. A., Mahadevan, S., & Marchalleck, N. (1999) · Management Science · doi:10.1287/mnsc.45.4.560 · Cited by 221 · see citing works
Self-citationRudkevich, A., Duckworth, M., & Rosen, R. J. (1998) · The Energy Journal · doi:10.5547/issn0195-6574-ej-vol19-no3-2 · Cited by 218 · see citing works
Learning rate schedules for faster stochastic gradient search
Darken, C., Chang, J. Y. T., & Moody, J. E. (2003) · doi:10.1109/nnsp.1992.253713 · Cited by 217 · see citing works
Learning Algorithms for Markov Decision Processes with Average Cost
Abounadi, J., Bertsekas, D. P., & Borkar, V. S. (2001) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012999361974 · Cited by 207 · see citing works
Market power in California electricity markets
Borenstein, S., Bushnell, J., Kahn, E. P., et al. (1995) · Utilities Policy · doi:10.1016/0957-1787(96)00005-7 · Cited by 186 · see citing works
Understanding how market power can arise in network competition: a game theoretic approach
Berry, C. A., Hobbs, B. F., Meroney, W. A., et al. (1999) · Utilities Policy · doi:10.1016/s0957-1787(99)00016-8 · Cited by 173 · see citing works
GOSAVII, A., Bandla, N., & Das, T. K. (2002) · IIE Transactions · doi:10.1080/07408170208928908 · Cited by 138 · see citing works
Self-citationEvaluating Individual Market Power in Electricity Markets via Agent-Based Simulation
Bunn, D. W., & Oliveira, F. S. (2003) · Annals of Operations Research · doi:10.1023/a:1023399017816 · Cited by 123 · see citing works
Short-Term Electricity Market Auction Game Analysis: Uniform and Pay-as-Bid Pricing
Son, Y. S., Baldick, R., Lee, K., et al. (2004) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2004.836192 · Cited by 111 · see citing works
(2000) · Applied Physics Letters · doi:10.1162/jmlr.2003.4.6.1039 · Cited by 108 · see citing works
Alternative Models of Restructured Electricity Systems, Part 1: No Market Power
Boucher, J., & Smeers, Y. (2001) · Operations Research · doi:10.1287/opre.49.6.821.10017 · Cited by 93 · see citing works
Electricity market design: the good, the bad, and the ugly
Cramton, P. (2003) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2003.1173866 · Cited by 92 · see citing works
Auction Design in Day-Ahead Electricity Markets
Contreras, J., Candiles, O., de la Fuente, J. I., et al. (2001) · IEEE Power Engineering Review · doi:10.1109/mper.2001.4311283 · Cited by 66 · see citing works
The Efficiency of Multi-Unit Electricity Auctions
Elmaghraby, W. J., & Oren, S. S. (1999) · The Energy Journal · doi:10.5547/issn0195-6574-ej-vol20-no4-4 · Cited by 59 · see citing works
Solving three-player games by the matrix approach with application to an electric power market
Lee, K., & Baldick, R. (2003) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2003.818744 · Cited by 58 · see citing works
Pay-as-Bid versus Marginal Pricing–Part II: Market Behavior Under Strategic Generator Offers
Ren, Y., & Galiana, F. D. (2004) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2004.835650 · Cited by 52 · see citing works
Application of actor-critic learning algorithm for optimal bidding problem of a genco
Gajjar, G. R., Khaparde, S. A., Nagaraju, P., et al. (2003) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2002.807041 · Cited by 51 · see citing works
Computing Cournot equilibria in two settlement electricity markets with transmission constraint
Yao, J., Oren, S. S., & Adler, I. (2004) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2004.1265176 · Cited by 47 · see citing works
Auction design in day-ahead electricity markets
Contreras, J., Candiles, O., Fuente, J. I. D. L., et al. (2001) · IEEE Transactions on Power Systems · doi:10.1109/59.910785 · Cited by 43 · see citing works
Market power and price volatility in restructured markets for electricity
Mount, T. D. (2003) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.1999.772841 · Cited by 38 · see citing works
Competition in Electricity Spot Markets. Economic Theory and International ExperiencePreprint
von der Fehr, N.-H. M., & Harbord, D. (1998) · RePEc: Research Papers in Economics · Cited by 35 · see citing works
A reinforcement learning approach to stochastic business games
Ravulapati, K. K., Rao, J., & Das, T. K. (2004) · IIE Transactions · doi:10.1080/07408170490278698 · Cited by 33 · see citing works
Self-citationDesigning Electricity Auctions: Uniform, Discriminatory and Vickrey ¤
Fabra, N., von der Fehr, N.-H. M., & Harbord, D. (2003) · Cited by 32 · see citing works
Spear, S. E. (2003) · Journal of Economic Theory · doi:10.1016/s0022-0531(03)00017-6 · Cited by 31 · see citing works
A Stochastic Game Approach for Modeling Wholesale Energy Bidding in Deregulated Power Markets
Ragupathi, R., & Das, T. K. (2004) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2004.825910 · Cited by 31 · see citing works
Self-citationEquilibrium market power modeling for large scale power systems
Hobbs, B. F., Helman, U., & Pang, J. (2001) · doi:10.1109/pess.2001.970095 · Cited by 23 · see citing works
Tuning of discretization in bimatrix game approach to power system market analysis
Lee, K., & Baldick, R. (2003) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2002.807067 · Cited by 22 · see citing works
Market power in generation markets
David, A. K. (2000) · International Conference on Advances in Power System Control, Operation and Management (APSCOM) · doi:10.1049/cp:20000400 · Cited by 20 · see citing works
Identifying the potential for market power in electric power systems in real-time
Kian, A. R., Thomas, R. J., Zimmerman, R. D., et al. (2004) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2004.1265178 · Cited by 16 · see citing works
Market Size, Market Structure & Market Power in the Irish Electricity IndustryPreprint
McCarthy, N. (2005) · RePEc: Research Papers in Economics · Cited by 11 · see citing works
Learning Average Reward Irreducible Stochastic Games: Analysis and Applications
Li, J. (2003) · Digital Commons - University of South Florida (University of South Florida) · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F. (2020) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000552 · Cited by 460 · see citing works
Cheng, L., & Yu, T. (2019) · International Journal of Energy Research · doi:10.1002/er.4333 · Cited by 380 · see citing works
Glavic, M., Fonteneau, R., & Ernst, D. (2017) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2017.08.1217 · Cited by 241 · see citing works
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works
- Authors
- Nanduri, V., & Das, T. K.
- Affiliations
- University of South Florida (US)
- ORCID
- Das, Tapas K.
- Venue
- IEEE Transactions on Power Systems
- Published
- 2007-02
- Volume
- 22
- Issue
- 1
- Pages
- 85-95
- ISSN
- 0885-8950 (ISSN-L)
- OpenAlex
- W2138620854
- Cited by
- 168
- References
- 46
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 8 · 2023: 6 · 2022: 6 · 2021: 8 · 2020: 12 · 2019: 43 · 2018: 7 · 2017: 4 · 2016: 9 · 2015: 5 · 2014: 2 · 2013: 8 · 2012: 6
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Power Systems (publishedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
- Social Sciences › Decision Sciences › Management Science and Operations Research › Auction Theory and Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Keywords
- market power, day-ahead market, reinforcement learning, bidding strategy, uniform-price auctions, discriminatory auctions, second-price auctions, Lerner index
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review