Q-Probabilistic Routing in Wireless Sensor Networks
Abstract
Unpredictable topology changes, energy constraints and link unreliability make the information transmission a challenging problem in wireless sensor networks (WSN). Taking some ideas from machine learning methods, we propose a novel geographic routing algorithm for WSN, named Q-probabilistic routing (Q-PR), that makes intelligent routing decisions from the delayed reward of previous actions and the local interaction among neighbor nodes, by using reinforcement learning and a Bayesian decision model. Moreover, by considering the message importance embedded in the message itself routing decisions can be adapted to traffic importance. Experimental results show that Q-PR becomes a routing policy that, as a function of the message importance, achieves a trade-off among the expected number of retransmissions (ETX), the successful delivery rate and the network lifetime.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Q-PR preserves the lifetime of the WSN better, 3× and 4× more than GPSR and EFE respectively.
“Thus the Q-PR algorithm preserves better the lifetime of the WSN (3× and 4× more than GPSR and EFE respectively).”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Q-PR as well as EFE outperform GPSR in terms of successful delivery rate, with over 98% against 75.66%.
“Results show that Q-PR as well as EFE outperform GPSR in terms of successful delivery rate (over 98% against 75.66%).”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Q-PR shows a lower number of retransmission retries and acknowledgements, on average over 50% and 40% less than GPSR and EFE respectively.
“Moreover, Q-PR shows lower number of retransmission retries and acknowledgements (on average over 50% and 40% less than GPSR and EFE respectively).”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Arroyo-Valles et al. propose Q-PR, a localization-aware routing scheme for WSNs that applies Q-learning to achieve a trade-off between packet delivery rate, expected transmission count, and network lifetime.
“Arroyo-Valles et al. [24] propose Q-probabilistic routing (Q-PR), a localization-aware routing scheme for WSNs that applies Q-learning to achieve a trade-off between packet delivery rate, expected transmission count (ETX), and network lifetime.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Q-PR requires each node to maintain locally a number of information regarding each of its neighbors, which hampers the scalability of the approach.
“However the algorithm requires that each node maintains locally a number of information regarding each of its neighbors.These include the distance between the nodes, the distance of the neighbor to the sink, the delivery probability between nodes, the estimated residual energy at the neighboor, and the 2-hop neighboors.This hampers the scalability of the approach.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Q-PR achieves up to 5× higher delivery rate compared to directed diffusion.
“Compared to directed diffusion: • up to 5× higher delivery rate • ≈ 20% lower overhead Q-PR [24]”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
In a simulated multi-hop network with 2 sources transmitting videos to the same destination, reducing the frequency of message exchange by 95% (from every 1ms to every 20ms) increases goodput and effective data rate by over 40%.
“Results show that by reducing the frequency of message exchange by 95% (from every 1ms to every 20ms), the goodput and effective data rate are increased by over 40%”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Q-PR achieves approximately 20% lower overhead compared to directed diffusion.
“Compared to directed diffusion: • up to 5× higher delivery rate • ≈ 20% lower overhead Q-PR [24]”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Reducing the frequency of message exchange by 95% increases video quality (PSNR) by 10%.
“the video quality, calculated in terms of peak signal-to-noise ratio (PSNR), is increased by 10%”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Reducing the frequency of message exchange by 95% only slightly affects convergence time (1∼2sec).
“The convergence time seems to be only slightly affected (1 ∼ 2sec).”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Utility functions such as QoS provisioning, load balancing and network lifetime maximization require more information to be disseminated at the cost of increased complexity and communication overhead.
“utility functions such as QoS provisioning, load balancing and network lifetime maximization, as in Q-PR [24], QELAR [197,277], require more information to be disseminated at the cost of an increased complexity and communication overhead.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [24]
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Karp, B., & Kung, H. T. (2000) · Proceedings of the 28th Annual International Conference on Mobile Computing And Networking · doi:10.1145/345910.345953 · Cited by 7,077 · see citing works
Detection, Estimation, and Modulation Theory, Part I
Trees, H. L. V. (2001) · John Wiley & Sons, Inc. eBooks · doi:10.1002/0471221082 · Cited by 3,923 · see citing works
A survey on routing protocols for wireless sensor networks
Akkaya, K., & Younis, M. (2003) · Ad Hoc Networks · doi:10.1016/j.adhoc.2003.09.010 · Cited by 3,500 · see citing works
Biswas, S., & Morris, R. (2005) · ACM SIGCOMM Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM) · doi:10.1145/1080091.1080108 · Cited by 1,357 · see citing works
Routing with Guaranteed Delivery in Ad Hoc Wireless Networks
Bose, P. K., Morin, P., Stojmenović, I., et al. (2001) · Wireless Networks · doi:10.1023/a:1012319418150 · Cited by 1,204 · see citing works
Opportunistic routing in multi-hop wireless networks
Biswas, S., & Morris, R. (2004) · ACM SIGCOMM Computer Communication Review · doi:10.1145/972374.972387 · Cited by 675 · see citing works
Position-based routing in ad hoc networks
Stojmenović, I. (2002) · IEEE Communications Magazine · doi:10.1109/mcom.2002.1018018 · Cited by 653 · see citing works
Medium Access Control in Wireless Sensor Networks
Ye, W., & Heidemann, J. (2006) · Springer eBooks · doi:10.1007/978-1-4020-7884-2_4 · Cited by 262 · see citing works
Localized positioning in ad hoc networks
Niculescu, D., & Nath, B. (2003) · Ad Hoc Networks · doi:10.1016/s1570-8705(03)00005-2 · Cited by 141 · see citing works
Stojmenović, I., Nayak, A. R., & Kuruvila, J. (2005) · IEEE Communications Magazine · doi:10.1109/mcom.2005.1404603 · Cited by 126 · see citing works
Mobilized ad-hoc networks: a reinforcement learning approach
Chang, Y.-H., Ho, T., & Kaelbling, L. P. (2004) · International Conference on Autonomic Computing · doi:10.1109/icac.2004.1301369 · Cited by 94 · see citing works
When Does Opportunistic Routing Make Sense?
Shah, R. C., Wiethölter, S., Wolisz, A., et al. (2005) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2005.90 · Cited by 89 · see citing works
On loss functions which minimize to conditional expected values and posterior probabilities
Miller, J. W. V., Goodman, R. M., & Smyth, P. (1993) · IEEE Transactions on Information Theory · doi:10.1109/18.243457 · Cited by 68 · see citing works
Stojmenović, I., Nayak, A. R., Kuruvila, J., et al. (2004) · Computer Communications · doi:10.1016/j.comcom.2004.07.040 · Cited by 61 · see citing works
Localized positioning in ad hoc networks
Niculescu, D., & Nath, B. K. (2003) · doi:10.1109/snpa.2003.1203355 · Cited by 48 · see citing works
Message-initiated constraint-based routing for wireless ad-hoc sensor networks
Zhang, Y., & Fromherz, M. P. J. (2004) · International Symposium on VLSI Technology, Systems and Applications (VLSI-TSA) · doi:10.1109/ccnc.2004.1286943 · Cited by 40 · see citing works
Confidence based dual reinforcement Q-routing: an adaptive online network routing algorithm
Kumar, S., & Mukkulainen, R. (1999) · Cited by 34 · see citing works
Smart Routing with Learning-Based QoS-Aware Meta-strategies
Zhang, Y., Fromherz, M. P. J., & Kuhn, L. D. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30193-6_30 · Cited by 21 · see citing works
A Bayesian Decision Model for Intelligent Routing in Sensor Networks
Arroyo-Valles, R., Marques, A. G., Díaz, J. J. V., et al. (2006) · International Symposium on Wireless Communication Systems · doi:10.1109/iswcs.2006.4362268 · Cited by 16 · see citing works
Self-citationStojmenović, M., & Nayak, A. R. (2006) · Computer Communications · doi:10.1016/j.comcom.2006.02.013 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
Machine learning in wireless sensor networks: Algorithms, strategies, and applications
Alsheikh, M. A., Lin, S., Niyato, D., & Tan, H.-P. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2320099 · Cited by 965 · see citing works
Fadlullah, Z. M., Tang, F., Mao, B., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2707140 · Cited by 810 · see citing works
- Authors
- Arroyo-Valles, R., Alaiz-Rodriguez, R., Guerrero-Curieses, A., & Cid-Sueiro, J.
- Affiliations
- Universidad Rey Juan Carlos (ES)
- Venue
- 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information
- Publisher
- IEEE
- Published
- 2007
- Pages
- 1-6
- OpenAlex
- W2532589578
- Cited by
- 65
- References
- 21
- Citations by year
- 2024: 2 · 2023: 5 · 2022: 2 · 2021: 4 · 2020: 9 · 2019: 9 · 2018: 6 · 2017: 4 · 2016: 5 · 2015: 1 · 2014: 3 · 2013: 4 · 2012: 1
- FWCI
- 1.62
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Intelligent Sensors, Sensor Networks and Information Processing (publishedVersion)
- Funders
- Universidad Carlos III de Madrid
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Energy Efficient Wireless Sensor Networks
- Physical Sciences › Computer Science › Computer Networks and Communications › Mobile Ad Hoc Networks
- Physical Sciences › Computer Science › Computer Networks and Communications › Security in Wireless Sensor Networks
- Keywords
- wireless sensor networks, geographic routing, reinforcement learning, Bayesian decision model, expected transmission time, retransmission, network lifetime, energy constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
