CoChat

Comparative analysis of Q-learning, SARSA, and deep Q-network for microgrid energy management

Open access
Ramesh, Sreyas, N, Sukanth B, Sathyavarapu, Sri Jaswanth, Sharma, Vishwash, A. A., Nippun Kumaar, Khanna, Manju · 2025 · Scientific Reports

The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.

Abstract

The growing integration of renewable energy sources within microgrids necessitates innovative approaches to optimize energy management. While microgrids offer advantages in energy distribution, reliability, efficiency, and sustainability, the variable nature of renewable energy generation and fluctuating demand pose significant challenges for optimizing energy flow. This research presents a novel application of Reinforcement Learning (RL) algorithms-specifically Q-Learning, SARSA, and Deep Q-Network (DQN)-for optimal energy management in microgrids. Utilizing the PyMGrid simulation framework, this study not only develops intelligent control strategies but also integrates advanced mathematical control techniques, such as Model Predictive Control (MPC) and Kalman filters, within the Markov Decision Process (MDP) framework. The innovative aspect of this research lies in its comparative analysis of these RL algorithms, demonstrating that DQN outperforms Q-Learning and SARSA by 12% and 30%, respectively, while achieving a remarkable 92% improvement over scenarios without an RL agent. This study addresses the unique challenges of energy management in microgrids and provides practical insights into the application of RL techniques, thereby contributing to the advancement of sustainable energy solutions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-024-83625-8

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Overview of the proposed system.
Fig. 1 Overview of the proposed system.CC BY-NC-ND · PubMed Central
Fig. 2 Deep-Q network model.
Fig. 2 Deep-Q network model.CC BY-NC-ND · PubMed Central
Fig. 3 Learning curve of the algorithms.
Fig. 3 Learning curve of the algorithms.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 20 references

Research works printed but not linked: 2

  1. Somasundaran, N., Radhika, N. & Venkataraman, V. Smart grid test bed based on GSM. Proc. Eng. (2012).
  2. Sowmya Reddy, V. S., Chandan, K., Nimmy, P., Smitha, T. V. & Nagaraja, K. V. An efficient machine learning model for smart grid stability prediction in our prestigious conference: International Conference on Emerging Technologies in Engineering and Science (ICETES) (2023).

Cited by

See all 36

Metadata

Authors
Ramesh, S., N, S. B., Sathyavarapu, S. J., Sharma, V., A. A., N. K., & Khanna, M.
Affiliations
Amrita Vishwa Vidyapeetham (IN)
Venue
Scientific Reports
Published
2025-01-03
Volume
15
Issue
1
Pages
694-694
ISSN
2045-2322 (ISSN-L)
OpenAlex
W4406024085
Cited by
35
References
20
Citations by year
2026: 20 · 2025: 15
FWCI
12.86
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Microgrid Control and Optimization
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
microgrid energy management, Q-learning, SARSA, deep Q-network, reinforcement learning, model predictive control, Kalman filter, Markov decision process, renewable energy integration, algorithm comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Comparative analysis of Q-learning, SARSA, and deep Q-network for microgrid energy management. https://app.cochat.ai/papers/10.1038/s41598-024-83625-8
The paper
Ramesh, S., N, S. B., Sathyavarapu, S. J., Sharma, V., A. A., N. K., & Khanna, M. (2025). Comparative analysis of Q-learning, SARSA, and deep Q-network for microgrid energy management. Scientific Reports. https://doi.org/10.1038/s41598-024-83625-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.