Multi-Agent Deep Reinforcement Learning for Scheduling of Energy Storage System in Microgrids

Abstract

Efficient scheduling of Energy Storage Systems (ESS) within microgrids has emerged as a critical issue to ensure energy cost reduction, peak shaving, and battery health management. For ESS scheduling, both single-agent and multi-agent deep reinforcement learning (DRL) approaches have been explored. However, the former has suffered from scalability to include multiple objectives while the latter lacks comprehensive consideration of diverse user objectives. To defeat the above issues, in this paper, we propose a new DRL-based scheduling algorithm using a multi-agent proximal policy optimization (MAPPO) framework that is combined with Pareto optimization. The proposed model employs two independent agents: one is to minimize electricity costs and the other does charge/discharge switching frequency to account for battery degradation. The candidate actions generated by the agents are evaluated through Pareto dominance, and the final action is selected via scalarization-reflecting operator-defined preferences. The simulation experiments were conducted using real industrial building load and photovoltaic (PV) generation data under realistic South Korean electricity tariff structures. The comparative evaluations against baseline DRL algorithms (TD3, SAC, PPO) demonstrate that the proposed MAPPO method significantly reduces electricity costs while minimizing battery-switching events. Furthermore, the results highlight that the proposed method achieves a balanced improvement in both economic efficiency and battery longevity, making it highly applicable to real-world dynamic microgrid environments. Specifically, the proposed MAPPO-based scheduling achieved a total electricity cost reduction of 14.68% compared to the No-ESS case and achieved 3.56% greater cost savings than other baseline reinforcement learning algorithms.