Adaptive call admission control under quality of service constraints: a reinforcement learning solution
Abstract
We solve the adaptive call admission control (CAC) problem in multimedia networks via reinforcement learning (RL). The problem requires that network revenue be maximized while simultaneously meeting quality of service (QoS) constraints that forbid entry into certain states and use of certain actions. We show that RL provides a solution to this constrained semi-Markov decision problem and is able to earn significantly higher revenues than alternative heuristics. Unlike other model-based algorithms, RL does not require the explicit state transition models to solve the decision problems. This feature is very important if one considers large integrated service networks supporting a number of different service types, where the number of states is so large that model-based optimization algorithms are infeasible. Both packet-level and call-level QoS constraints are addressed, and both conservative and aggressive approaches to the QoS constraints are considered. Results are demonstrated on a single link and extended to routing on a multilink network.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tong et al. [446] formulate admission control with routing as a semi-MDP and leverage Q-learning to define policies for route selection, such that revenue is maximized and QoS requirements of requests are met.
“Tong et al. [446] formulate this problem as a semi-MDP, and leverage Q-learning to define policies for route selection, such that the revenue is maximized and QoS requirements of the requests are met.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [446]
The results of Tong et al. [446] show significant improvement over heuristic based algorithms.
“The results show significant improvement over heuristic based algorithms.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [446]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Wide area traffic: the failure of Poisson modeling
Paxson, V., & Floyd, S. (1995) · IEEE/ACM Transactions on Networking · doi:10.1109/90.392383 · Cited by 3,721 · see citing works
Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach
Boyan, J. A., & Littman, M. L. (1994) · Cited by 694 · see citing works
Reinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems
Singh, S. P., & Bertsekas, D. P. (1996) · Cited by 213 · see citing works
Multi-criteria Reinforcement Learning
Gábor, Z., Kalmár, Z., & Szepesvári, C. (1998) · Cited by 202 · see citing works
A Q-learning-based dynamic channel assignment technique for mobile communication systems
Nie, J., & Haykin, S. (1999) · IEEE Transactions on Vehicular Technology · doi:10.1109/25.790549 · Cited by 182 · see citing works
ATM communications network control by neural networks
Hiramatsu, A. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80211 · Cited by 152 · see citing works
Constrained Semi-Markov decision processes with average rewards
Feinberg, E. A. (1994) · Mathematical Methods of Operations Research · doi:10.1007/bf01435458 · Cited by 97 · see citing works
Time-average optimal constrained semi-Markov decision processes
Beutler, F. J., & Ross, K. W. (1986) · Advances in Applied Probability · doi:10.2307/1427303 · Cited by 84 · see citing works
Robust dynamic admission control for unified cell and call QoS in statistical multiplexers
Mitra, D., Reiman, M. I., & Wang, J. (1998) · IEEE Journal on Selected Areas in Communications · doi:10.1109/49.700906 · Cited by 72 · see citing works
Call admission and routing in multi-service loss networks
Dziong, Z., & Mason, L. G. (1994) · IEEE Transactions on Communications · doi:10.1109/tcomm.1994.583415 · Cited by 68 · see citing works
ATM Network Resource Management
Dziong, Z. (1997) · Espace ÉTS (ETS) · Cited by 59 · see citing works
Adaptive control of constrained Markov chains
Altman, E., & Schwartz, A. (1991) · IEEE Transactions on Automatic Control · doi:10.1109/9.75103 · Cited by 35 · see citing works
Markov Decision Algorithms for Dynamic Routing
Krishnan, K. R. (1990) · Cited by 34 · see citing works
Markov decision algorithms for dynamic routing (telephone networks)
Krishnan, K. R. (1990) · IEEE Communications Magazine · doi:10.1109/35.60368 · Cited by 30 · see citing works
Reinforcement Learning for Call Admission Control and Routing in Integrated Service Networks
Marbach, P., Mihatsch, O., Schulte, M., & Tsitsiklis, J. N. (1997) · Cited by 25 · see citing works
Verma, S., Pankaj, R. K., & Leon‐Garcia, A. (1998) · Computer Communications · doi:10.1016/s0140-3664(97)00169-2 · Cited by 18 · see citing works
Marbach, P., Tsitsiklis, J. N., & Systems., D. (1997) · Cited by 13 · see citing works
Adaptive Access Control Applied to Ethernet Data
Brown, T. X. (1996) · neural information processing systems · Cited by 7 · see citing works
Self-citationA neuro-dynamic programming approach to admission control in ATM networks: the single link case
Marbach, P., & Tsitsiklis, J. N. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1997.599583 · Cited by 6 · see citing works
Estimating loss rates in an integrated services network by neural networks
Tong, H., & Brown, T. X. (2002) · Globecom · doi:10.1109/glocom.1998.775690 · Cited by 5 · see citing works
Self-citationTong, H., & Brown, T. X. (1999) · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
A Self-Learning Call Admission Control Scheme for CDMA Cellular Networks
Liu, D., Zhang, Y., & Zhang, H. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.853408 · Cited by 162 · see citing works
A Model Based RL Admission Control Algorithm for Next Generation Networks
Mignanti, S., Giorgio, A. D., & Suraci, V. (2009) · 2009 Eighth International Conference on Networks · doi:10.1109/icn.2009.39 · Cited by 9 · see citing works
- Authors
- Tong, H., & Brown, T. X.
- Affiliations
- University of Colorado Boulder (US)
- ORCID
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2000-02
- Volume
- 18
- Issue
- 2
- Pages
- 209-221
- ISSN
- 0733-8716 (ISSN-L)
- OpenAlex
- W2103588946
- Cited by
- 85
- References
- 24
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 5 · 2022: 2 · 2021: 2 · 2020: 2 · 2019: 4 · 2018: 2 · 2014: 1 · 2013: 1 · 2012: 1
- FWCI
- 9.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Journal on Selected Areas in Communications (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Traffic and Congestion Control
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Communication Networks Research
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Network Optimization
- Keywords
- quality of service constraints, reinforcement learning, multimedia networks, semi-Markov decision process, network revenue maximization, integrated services networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
