Deep Blue: A Fuzzy Q-Learning Enhanced Active Queue Management Scheme
Abstract
Although RED has been widely used with TCP, however it has several known drawbacks [1]. The BLUE algorithm that benefits from a different structure has tried to compensate some of them in a successful way [2]. A quick review on active queue management algorithms from the very beginning indicates that most of them tried to improve classic algorithms. Some of them use network traffic history to achieve more flexibility and prediction ability while others use algorithms such as fuzzy logic to address scalability problem and high input load. Our proposed approach benefits from both: Using fuzzy logic to deal with high input load and embedding expert knowledge into the algorithm while optimizing router decisions with reinforcement learning fed by network traffic history. We call this approach "DEEP BLUE" as is consist of an improved version of BLUE algorithm. Derived from BLUE, our algorithm uses packet drop rate and link idle events to manage congestion. Our experiments using OPNET simulator shows that this scheme works faster and more efficient than original BLUE.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These AQM schemes apply different supervised techniques for TSF and reinforcement-based methods for deducing the increment in the packet drop probability.
“these AQM schemes apply different supervised techniques for TSF [160, 179, 212, 498] and reinforcementbased methods for deducing the increment in the packet drop probability [298, 427, 428, 485, 499].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [298]
DEEP BLUE improves BLUE in terms of queue stabilization and dropping policy.
“DEEP BLUE improves BLUE in terms of queue stabilization and dropping policy.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [298]
DEEP BLUE only generates a slight surplus of storage and computational overhead over BLUE, though no evaluation results are reported.
“DEEP BLUE only generates a slight surplus of storage and computational overhead over BLUE, though no evaluation results are reported.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [298]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
The BLUE active queue management algorithms
Feng, W.-C., Shin, K. G., Kandlur, D. D., & Saha, D. (2002) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2002.801399 · Cited by 576 · see citing works
Fuzzy inference system learning by reinforcement methods
Jouffe, L. (1998) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/5326.704563 · Cited by 385 · see citing works
Refinement of Approximate Reasoning-based Controllers by Reinforcement Learning
Berenji, H. R., & Software, S. (1991) · Elsevier eBooks · doi:10.1016/b978-1-55860-200-7.50097-0 · Cited by 46 · see citing works
A fuzzy logic-based AQM for real-time traffic over internet
Aoul, Y. H., Mehaoua, A., & Skianis, C. A. (2007) · Computer Networks · doi:10.1016/j.comnet.2007.06.007 · Cited by 39 · see citing works
Fuzzy logic controlled RED: congestion control in TCP/IP differentiated services networks
Chrysostomou, C., Pitsillides, A., Rossides, L., & Şekercioğlu, Y. A. (2003) · Soft Computing · doi:10.1007/s00500-002-0248-9 · Cited by 37 · see citing works
PSO-PID: a novel controller for AQM routers
Wang, X., Wang, Y., Zhou, H. X., & Huai, X. (2006) · IFIP International Conference on Wireless and Optical Communications Networks (WOCN) · doi:10.1109/wocn.2006.1666682 · Cited by 26 · see citing works
Stabilizing RED using a Fuzzy Controller
Sun, J., Zukerman, M., & Palaniswami, M. S. (2007) · International Conference on Communications · doi:10.1109/icc.2007.52 · Cited by 19 · see citing works
Multi-objective Particle Swarm Optimization for Fuzzy Logic Based Active Queue Management
Nyirenda, C., & Dawoud, D. S. (2006) · Proceedings of ... IEEE International Conference on Fuzzy Systems · doi:10.1109/fuzzy.2006.1682010 · Cited by 18 · see citing works
Design of an active queue management algorithm based fuzzy logic decision
Fan, Y., Fengyuan, R., & Lin, C. (2004) · doi:10.1109/icct.2003.1209086 · Cited by 14 · see citing works
α_ SNFAQM: an active queue management mechanism using neurofuzzy prediction
Zhani, M. F., Elbiaze, H., & Kamoun, F. (2007) · 2007 12th IEEE Symposium on Computers and Communications · doi:10.1109/iscc.2007.4381596 · Cited by 11 · see citing works
Fuzzy DS RED An Intelligent Active Queue Management Scheme for TCP / IP Diff-Serv.
Subasree, S., & Ravichandran, K. S. (2004) · International Conference on Computational Intelligence · Cited by 8 · see citing works
Jahanshahi, M., & Meybodi, M. R. (2007) · International Conference on Mechatronics and Automation · doi:10.1109/icma.2007.4304102 · Cited by 7 · see citing works
Improving RED by a Neuron Controller
Sun, J., & Zukerman, M. (2007) · Springer eBooks · doi:10.1007/978-3-540-72990-7_40 · Cited by 6 · see citing works
RBF-PID Based Adaptive Active Queue Management Algorithm for TCP Network
Wang, J., Zhi-wei, G., & Yan-tai, S. (2007) · International Conference on Control and Automation · doi:10.1109/icca.2007.4376341 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
Active Queue Management: A Survey
Adams, R. V. (2012) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.082212.00018 · Cited by 283 · see citing works
A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot
Konar, A., Chakraborty, I. G., Singh, S. J., Jain, L. C., & Nagar, A. K. (2013) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmca.2012.2227719 · Cited by 273 · see citing works
When machine learning meets congestion control: A survey and comparison
Jiang, H., Li, Q., Jiang, Y., Shen, G., Sinnott, R., Tian, C., & Xu, M. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108033 · Cited by 107 · see citing works
In-Network Machine Learning Using Programmable Network Devices: A Survey
Zheng, C., Hong, X., Ding, D., Vargaftik, S., Ben-Itzhak, Y., & Zilberman, N. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3344351 · Cited by 90 · see citing works
Khatari, M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., & Alsalem, M. A. (2019) · International Journal of Information Technology & Decision Making · doi:10.1142/s0219622019300039 · Cited by 64 · see citing works
An Advance Q Learning (AQL) Approach for Path Planning and Obstacle Avoidance of a Mobile Robot
Chakraborty, A., & Banerjee, J. S. (2013) · International Journal of Intelligent Mechatronics and Robotics · doi:10.4018/ijimr.2013010105 · Cited by 24 · see citing works
An Improved Q-learning Algorithm for Path-Planning of a Mobile Robot
KDas, P., Mandhata, S. C., Behera, H. S., & Patro, S. N. (2012) · International Journal of Computer Applications · doi:10.5120/8073-1468 · Cited by 22 · see citing works
Controlling Delay at the Router Buffer using Modified Random Early Detection
Abu-Shareha, A. A. (2019) · International journal of Computer Networks & Communications · doi:10.5121/ijcnc.2019.11604 · Cited by 21 · see citing works
Liu, C.-T., & Hsu, R. C. (2011) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2011.6119679 · Cited by 14 · see citing works
Linear Random Early Detection for Congestion Control at the Router Buffer
Abu-Shareha, A. A., Al-kasasbeh, B. M., Shambour, Q. Y., Abualhaj, M. M., Alsharaiah, M. A., & Al-Khatib, S. N. (2022) · Informatica · doi:10.31449/inf.v46i5.3966 · Cited by 6 · see citing works
Machine learning controller for data rate management in science DMZ networks
Caicedo, C. V., Kfoury, E. F., Gomez, J., Pezoa, J. E. E., Figueroa, M. E., & Crichigno, J. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.110237 · Cited by 5 · see citing works
Data-driven modeling and transportation data analytics
Wei, D. (2014) · ThinkTech (Texas Tech University) · Cited by 2 · see citing works
Latency and Throughput Optimization in Modern Networks: A Comprehensive SurveyPreprint
Mirzaeinia, A., Mirzaeinia, M., & Rezgui, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.03715 · Cited by 11 · see citing works
Figueroa, M., & Pezoa, J. E. E. (2023) · doi:10.29393/tdudec-15vc1ef15 · Cited by 1 · see citing works
A new algorithm to track dynamic goal position in Q-learning
Mitra, S., Banerjee, D., Konar, A., & Janarthanan, R. (2012) · International Conference on Hybrid Intelligent Systems (HIS) · doi:10.1109/his.2012.6421311
Mohammed, S. J., & Ali, M. H. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-23314-1_22
- Authors
- Masoumzadeh, S. S., Taghizadeh, G., Meshgi, K., & Shiry, S.
- Affiliations
- Qazvin Islamic Azad University (IR)
- Amirkabir University of Technology (IR)
- Venue
- 2009 International Conference on Adaptive and Intelligent Systems
- Publisher
- IEEE
- Published
- 2009-09
- Pages
- 43-48
- OpenAlex
- W2136962080
- Cited by
- 17
- References
- 16
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 2 · 2022: 1 · 2021: 1 · 2020: 1 · 2019: 2 · 2018: 1 · 2014: 1 · 2013: 2 · 2012: 3
- FWCI
- 1.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Adaptive and Intelligent Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Traffic and Congestion Control
- Physical Sciences › Computer Science › Computer Networks and Communications › Software-Defined Networks and 5G
- Physical Sciences › Engineering › Control and Systems Engineering › Traffic control and management
- Keywords
- active queue management, fuzzy Q-learning, red, congestion control, packet loss rate, reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
