Smart Congestion Control for Delay- and Disruption Tolerant Networks
Abstract
In this paper, we propose a novel congestion control framework for delay- and disruption tolerant networks (DTNs). The proposed framework, called Smart-DTN-CC, adjusts its operation automatically as a function of the dynamics of the underlying net- work. It employs reinforcement learning, a machine learning technique known to be well suited to problems in which the environment, in this case the network, plays a crucial role; yet, no prior knowledge about the target environment can be assumed, i.e., the only way to acquire information about the environment is to interact with it through continuous online learning. Smart-DTN-CC nodes get input from the environment (e.g., its buffer occupancy, set of neighbors, etc), and, based on that information, choose an action to take from a set of possible actions. Depending on an action's effectiveness in controlling congestion, it will be given a reward. Smart-DTN-CC's goal is to maximize the over- all reward which translates to minimizing congestion. To our knowledge, Smart-DTN-CC is the first DTN congestion control framework that has the ability to automatically and continuously adapt to the dynamics of the target environment. As demonstrated by our experimental evaluation, Smart-DTN- CC is able to consistently outperform existing DTN congestion control mechanisms under a wide range of network conditions and characteristics.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Smart-DTN-CC achieves higher delivery ratio and significantly lower delay than existing DTN congestion control mechanisms such as CCC and SR.
“Simulations in the Opportunistic Network Environment (ONE) tool show that Smart-DTN-CC achieves higher delivery ratio and significantly lower delay than existing DTN congestion control mechanisms, such as CCC [265] and SR [406].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [412]
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,406 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
A survey of mobility models for ad hoc network research
Camp, T. K., Boleng, J. L., & Davies, V. (2002) · Wireless Communications and Mobile Computing · doi:10.1002/wcm.72 · Cited by 4,643 · see citing works
The ONE simulator for DTN protocol evaluation
Keränen, A., Ott, J., & Kärkkäinen, T. (2009) · International ICST Conference on Simulation Tools and Techniques (SIMUTools) · doi:10.4108/icst.simutools2009.5674 · Cited by 2,326 · see citing works
Outside the Closed World: On Using Machine Learning for Network Intrusion Detection
Sommer, R., & Paxson, V. (2010) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2010.25 · Cited by 2,316 · see citing works
Sutton, R. S. (1990) · Elsevier eBooks · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,348 · see citing works
Automatic programming of behavior-based robots using reinforcement learning
Mahadevan, S., & Connell, J. H. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90058-6 · Cited by 591 · see citing works
Sammut, C., & Webb, G. I. (2010) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_447 · Cited by 400 · see citing works
Rational and convergent learning in stochastic games
Bowling, M., & Veloso, M. (2001) · Cited by 280 · see citing works
Autonomous helicopter control using reinforcement learning policy search methods
Bagnell, J. A., & Schneider, J. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2001.932842 · Cited by 278 · see citing works
Acquiring robot skills via reinforcement learning
(1994) · IEEE Control Systems · doi:10.1109/37.257890 · Cited by 206 · see citing works
Byrd, R. J., Steinhubl, S. R., Sun, J., Ebadollahi, S., & Stewart, W. F. (2013) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.12.005 · Cited by 130 · see citing works
A survey on congestion control for delay and disruption tolerant networks
Silva, A. P., Burleigh, S. C., Hirata, C. M., & Obraczka, K. (2014) · Ad Hoc Networks · doi:10.1016/j.adhoc.2014.07.032 · Cited by 106 · see citing works
Self-citationRetiring Replicants: Congestion Control for Intermittently-Connected Networks
Thompson, N. A., Nelson, S. C., Bakht, M., Abdelzaher, T., & Kravets, R. H. (2010) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2010.5461934 · Cited by 82 · see citing works
Alternative custodians for congestion control in delay tolerant networks
Seligman, M., Fall, K., & Mundur, P. (2006) · doi:10.1145/1162654.1162660 · Cited by 81 · see citing works
Doǧan, Ü., Edelbrunner, J., & Iossifidis, I. (2011) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2011.6181557 · Cited by 77 · see citing works
A machine learning approach to improve congestion control over wireless computer networks
Geurts, P., El Khayat, I., & Leduc, G. (2004) · Fourth IEEE International Conference on Data Mining (ICDM'04) · doi:10.1109/icdm.2004.10063 · Cited by 68 · see citing works
Store-and-Forward Performance in a DTN
Chuah, M.-C., Yang, P., Davison, B. D., & Cheng, L. (2006) · IEEE Vehicular Technology Conference · doi:10.1109/vetecs.2006.1682801 · Cited by 52 · see citing works
ARBR: Adaptive reinforcement-based routing for DTN
Elwhishi, A., Ho, P.-H., Naik, K., & Shihada, B. (2010) · 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications · doi:10.1109/wimob.2010.5645040 · Cited by 49 · see citing works
Mohammad, Y. F. O., & Nishida, T. (2015) · Advanced information and knowledge processing · doi:10.1007/978-3-319-25232-2_13 · Cited by 47 · see citing works
A machine learning framework for TCP round-trip time estimation
Arouche Nunes, B. A., Veenstra, K., Ballenthin, W., Lukin, S., & Obraczka, K. (2014) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/1687-1499-2014-47 · Cited by 43 · see citing works
Self-citationA Novel Congestion Control Strategy in Delay Tolerant Networks
Li, Y., Xinjian, C., Qilie, L., & You, X. (2010) · International Conference on Future Networks · doi:10.1109/icfn.2010.49 · Cited by 32 · see citing works
Evaluating a Reinforcement Learning Algorithm with a General Intelligence Test
Insa-Cabrera, J., Dowe, D. L., & Hernández‐Orallo, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-25274-7_1 · Cited by 27 · see citing works
On the potential of recommendation technologies for efficient content delivery networks
Kâafar, M. A., Berkovsky, S., & Donnet, B. (2013) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2500098.2500109 · Cited by 20 · see citing works
Heuristic Congestion Control for Message Deletion in Delay Tolerant Network
Leela-amornsin, L., & Esaki, H. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-14891-0_26 · Cited by 19 · see citing works
Adaptive data offloading in opportunistic networks through an actor-critic learning method
Valerio, L., Bruno, R., & Passarella, A. (2014) · doi:10.1145/2645672.2645676 · Cited by 18 · see citing works
Rating Network Paths for Locality-Aware Overlay Construction and Routing
Du, W., Liao, Y., Tao, N., Geurts, P., Fu, X., & Leduc, G. (2014) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2014.2337371 · Cited by 12 · see citing works
Evaluation of reinforcement learning techniques
Yadav, A. K., & Shrivastava, S. K. (2010) · doi:10.1145/1963564.1963578 · Cited by 9 · see citing works
Congestion control in disruption-tolerant networks
Silva, A. P., Burleigh, S. C., Hirata, C. M., & Obraczka, K. (2014) · ACM Workshop on Challenged Networks (CHANTS) · doi:10.1145/2645672.2645684 · Cited by 8 · see citing works
Self-citationNetwork State Estimation Using Smart Experts
Ahn, J.-S., Edalat, Y., & Obraczka, K. (2014) · International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) · doi:10.4108/icst.mobiquitous.2014.257949 · Cited by 4 · see citing works
Self-citationPerforming and making use of mobility prediction
François, J.-M. (2007) · Open Repository and Bibliography (University of Liège) · Cited by 2 · see citing works
DTN Congestion Control Unplugged
Silva, A. P., Burleigh, S. C., Hirata, C. M., & Obraczka, K. (2015) · ACM Workshop on Challenged Networks (CHANTS) · doi:10.1145/2799371.2799383 · Cited by 1 · see citing works
Self-citationA machine learning approach to problems in computer network performance analysis
Barford, P., & Mirza, M. (2012) · Cited by 1 · see citing works
Flagged first, then by citations
3 of the 39 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
A Deep Reinforcement Learning Perspective on Internet Congestion Control
Jay, N. H., Rotman, N. H., Godfrey, B., Schapira, M., & Tamar, A. (2019) · International Conference on Machine Learning · Cited by 222 · see citing works
A Survey on Recent Advances in Transport Layer Protocols
Polese, M., Chiariotti, F., Bonetto, E., Rigotto, F., Zanella, A., & Zorzi, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2932905 · Cited by 153 · see citing works
When machine learning meets congestion control: A survey and comparison
Jiang, H., Li, Q., Jiang, Y., Shen, G., Sinnott, R., Tian, C., & Xu, M. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108033 · Cited by 107 · see citing works
In-Network Machine Learning Using Programmable Network Devices: A Survey
Zheng, C., Hong, X., Ding, D., Vargaftik, S., Ben-Itzhak, Y., & Zilberman, N. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3344351 · Cited by 90 · see citing works
Reinforcement Learning Meets Wireless Networks: A Layering Perspective
Chen, Y., Liu, Y., Zeng, M., Saleem, U., Lu, Z., Wen, X., Jin, D., Han, Z., Jiang, T., & Li, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3025365 · Cited by 41 · see citing works
Internet Congestion Control via Deep Reinforcement LearningPreprint
Jay, N. H., Rotman, N. H., Godfrey, P. B., Schapira, M., & Tamar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03259 · Cited by 38 · see citing works
A View on Deep Reinforcement Learning in System OptimizationPreprint
Haj-Ali, A., Ahmed, N. K., Willke, T. L., Gonzalez, J. E., Asanović, K., & Stoica, I. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01275 · Cited by 39 · see citing works
Machine Learning for Broad-Sensed Internet Congestion Control and Avoidance: A Comprehensive Survey
Huang, H., Zhu, X., Bi, J., Cao, W., & Zhang, X. (2021) · IEEE Access · doi:10.1109/access.2021.3060287 · Cited by 20 · see citing works
A congestion control framework for delay- and disruption tolerant networks
Silva, A. P., Obraczka, K., Burleigh, S. C., Nogueira, J. M. S., & Hirata, C. M. (2019) · Ad Hoc Networks · doi:10.1016/j.adhoc.2019.101880 · Cited by 18 · see citing works
Self-citationMulti-Channel Joint Forecasting-Scheduling for the Internet of Things
Rodoplu, V., Nakıp, M., Qorbanian, R., & Eliiyi, D. T. (2020) · IEEE Access · doi:10.1109/access.2020.3038358 · Cited by 17 · see citing works
Machine learning controller for data rate management in science DMZ networks
Caicedo, C. V., Kfoury, E. F., Gomez, J., Pezoa, J. E. E., Figueroa, M. E., & Crichigno, J. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.110237 · Cited by 5 · see citing works
Farooq, F., & Bibi, N. (2018) · EAI Endorsed Transactions on Energy Web · doi:10.4108/eai.10-4-2018.154457 · Cited by 3 · see citing works
Machine Learning in Delay Tolerant Networks: Algorithms, Strategies, and Applications
(2019) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.a1009.1191s19 · Cited by 2 · see citing works
A., E., Salawu, S. I., & Yahaya, B. (2017) · International Journal of Computer Applications · doi:10.5120/ijca2017915424 · Cited by 1 · see citing works
Study on a Train- and Bus-based Delay-Tolerant Networks: Scheduled Mobility and Impact on Routing
Wibowo, A. U. A., Sulistyo, S., & Mustika, I. W. (2019) · International Conference of Artificial Intelligence and Information Technology (ICAIIT) · doi:10.1109/icaiit.2019.8834675 · Cited by 1 · see citing works
Figueroa, M., & Pezoa, J. E. E. (2023) · doi:10.29393/tdudec-15vc1ef15 · Cited by 1 · see citing works
Studying Semi-TCP and Its Application in Marine Internet
Zhou, L. S., Jiang, S., & Xiong, C.-L. (2018) · Future Internet · doi:10.3390/fi10060044
Glisic, S., & Lorenzo, B. (2022) · Wiley eBooks · doi:10.1002/9781119790327.ch7
A Survey on Reinforcement Learning Routing Algorithm Research in Delay/Disruption Tolerant Network
Kim, G., Song, C., & Cho, S. (2025) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc66702.2025.11388522
- Authors
- Silva, A. P., Obraczka, K., Burleigh, S., & Hirata, C. M.
- Affiliations
- Universidade Federal de Minas Gerais (BR)
- University of California, Santa Cruz (US)
- California Institute of Technology (US)
- Venue
- 2016 13th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 1-9
- OpenAlex
- W2548859221
- Cited by
- 20
- References
- 39
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 2 · 2022: 1 · 2021: 2 · 2020: 2 · 2019: 6 · 2018: 4 · 2017: 1
- FWCI
- 3.13
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Sensing, Communication, and Networking (SECON) (publishedVersion)
- Funders
- National Science Foundation
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Fundação de Amparo à Pesquisa do Estado de Minas Gerais
- Jet Propulsion Laboratory
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Opportunistic and Delay-Tolerant Networks
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Traffic and Congestion Control
- Physical Sciences › Computer Science › Computer Networks and Communications › Software-Defined Networks and 5G
- Keywords
- congestion control, delay tolerant networks, reinforcement learning, adaptive network control, online learning, buffer occupancy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
