Learning-Based and Data-Driven TCP Design for Memory-Constrained IoT
Abstract
Advances in wireless technology have resulted in pervasive deployment of devices of a high variability in form factors, memory and computational ability. The need for maintaining continuous connections that deliver data with high reliability necessitate re-thinking of conventional design of the transport layer protocol. This paper investigates the use of Q-learning in TCP cwnd adaptation during the congestion avoidance state, wherein the classical alternation of the window is replaced, thereby allowing the protocol to immediately respond to previously seen network conditions. Furthermore, it demonstrates how memory plays a critical role in building the exploration space, and proposes ways to reduce this overhead through function approximation. The superior performance of the learning-based approach over TCP New Reno is demonstrated through a comprehensive simulation study, revealing 33.8% and 12.1% improvement in throughput and delay, respectively, for the evaluated topologies. We also show how function approximation can be used to dramatically reduce the memory requirements of a learning-based protocol while maintaining the same throughput and delay.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some RL-based approaches for updating CWND are designed for resource-constrained networks including WANETs and IoT, while others address wider network architectures such as satellite, cellular, and data center networks.
“Some of these approaches are particularly designed for resourceconstrained networks, including WANETs [29,219,379,380] and IoT [271], while others address a wider range of network architectures [30,122,477], such as satellite, cellular, and data center networks.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [271]
Basic simulations in ns-3 reveal that FK-TCPLearning improves the throughput and delay of TCP-NewReno, while being marginally inferior to TCPLearning.
“Furthermore, basic simulations in ns-3 reveal that FK-TCPLearning improves the throughput and delay of TCP-NewReno, while being marginally inferior to TCPLearning.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [271]
FK-TCPLearning with a set of 100 prototypes needs only 1.2% (2.4KB) of the memory used by pure Q-learning TCPLearning (200KB) for storing 50,000 state-action pairs.
“Specifically, FK-TCPLearning with a set of 100 prototypes, needs only 1.2% (2.4KB) of the memory used by TCPLearning based on pure Q-learning (200KB), for storing 50,000 state-action pairs.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [271]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,406 · see citing works
TCP Vegas: end to end congestion avoidance on a global Internet
Brakmo, L. S., & Peterson, L. (1995) · IEEE Journal on Selected Areas in Communications · doi:10.1109/49.464716 · Cited by 1,707 · see citing works
FAST TCP: Motivation, Architecture, Algorithms, Performance
Wei, D. X., Jin, C., Low, S. H., & Hegde, S. (2006) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2006.886335 · Cited by 956 · see citing works
Residual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. C. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 948 · see citing works
Mascolo, S., Casetti, C., Gerla, M., Sanadidi, M. Y., & Wang, R. (2001) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/381677.381704 · Cited by 922 · see citing works
Fidelity and yield in a volcano monitoring sensor network
Werner-Allen, G., Lorincz, K., Johnson, J., et al. (2006) · Cited by 647 · see citing works
RFID Technology for IoT-Based Personal Healthcare in Smart Spaces
Amendola, S., Lodato, R., Manzari, S., Occhiuzzi, C., & Marrocco, G. (2014) · IEEE Internet of Things Journal · doi:10.1109/jiot.2014.2313981 · Cited by 608 · see citing works
CMAC: an associative neural network alternative to backpropagation
Miller, W. T., Glanz, F. H., & Kraft, L. G. (1990) · Proceedings of the IEEE · doi:10.1109/5.58338 · Cited by 347 · see citing works
PCC: Re-architecting Congestion Control for Consistent High PerformancePreprint
Dong, M., Li, Q., Zarchy, D., Godfrey, B., & Schapira, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7092 · Cited by 298 · see citing works
CitySense: An Urban-Scale Wireless Sensor Network and Testbed
Murty, R., Mainland, G., Rose, I., et al. (2008) · IEEE Conference on Technologies for Homeland Security (HST) · doi:10.1109/ths.2008.4534518 · Cited by 267 · see citing works
Winstein, K., & Balakrishnan, H. (2013) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2534169.2486020 · Cited by 229 · see citing works
A Machine Learning Approach to TCP Throughput Prediction
Mirza, M., Sommers, J., Barford, P., & Zhu, X. (2010) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2009.2037812 · Cited by 132 · see citing works
An experimental study of the learnability of congestion control
Sivaraman, A., Winstein, K., Thaker, P., & Balakrishnan, H. (2014) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2619239.2626324 · Cited by 96 · see citing works
Sparse Distributed Memories for On-Line Value-Based Reinforcement Learning
Ratitch, B., & Precup, D. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30115-8_33 · Cited by 54 · see citing works
An experimental study of the learnability of congestion control
Sivaraman, A., Winstein, K., Thaker, P., & Balakrishnan, H. (2014) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2740070.2626324 · Cited by 54 · see citing works
Jayaraj, A., Venkatesh, T., & Murthy, C. S. R. (2008) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsacocn.2008.033508 · Cited by 40 · see citing works
A Machine Learning Approach to End-to-End RTT Estimation and its Application to TCP
Nunes, B. A. A., Veenstra, K., Ballenthin, W., et al. (2011) · Proceedings/Proceedings - International Conference on Computer Communications and Networks · doi:10.1109/icccn.2011.6006098 · Cited by 28 · see citing works
Adaptive Kanerva-based function approximation for multi-agent systems
Wu, C. F., & Meleis, W. M. (2008) · doi:10.5555/1402821.1402872 · Cited by 14 · see citing works
Self-citationFuzzy Kanerva-based function approximation for reinforcement learning
Wu, C. F., & Meleis, W. M. (2009) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/wypu1988 · Cited by 10 · see citing works
Self-citationMulti-resolution state-space discretization for Q-Learning with pseudo-randomized discretization
Lampton, A. K., Valasek, J., & Kumar, M. (2010) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596516 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
- Authors
- Li, W., Zhou, F., Meleis, W., & Chowdhury, K.
- Affiliations
- Northeastern University (US)
- Venue
- 2016 International Conference on Distributed Computing in Sensor Systems (DCOSS)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 199-205
- OpenAlex
- W2508010543
- Cited by
- 29
- References
- 22
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 4 · 2022: 5 · 2021: 4 · 2020: 2 · 2019: 3 · 2018: 6 · 2017: 2
- FWCI
- 3.13
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) (publishedVersion)
- Funders
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Traffic and Congestion Control
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › Software-Defined Networks and 5G
- Keywords
- TCP congestion control, Q-learning, memory-constrained devices, Internet of Things, function approximation, memory cost reduction, TCP NewReno, throughput, delay
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
