The DOI matches its Crossref record, with a small difference in title or year; 39 of the 40 references it cites were found in a registry or an index.
A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments
Open accessAbstract
Fog/Edge computing is a novel computing paradigm supporting resource-constrained Internet of Things (IoT) devices by placement of their tasks on edge and/or cloud servers. Recently, several Deep Reinforcement Learning (DRL)-based placement techniques have been proposed in fog/edge computing environments, which are only suitable for centralized setups. The training of well-performed DRL agents requires manifold training data while obtaining training data is costly. Hence, these centralized DRL-based techniques lack generalizability and quick adaptability, thus failing to efficiently tackle application placement problems. Moreover, many IoT applications are modeled as Directed Acyclic Graphs (DAGs) with diverse topologies. Satisfying dependencies of DAG-based IoT applications incur additional constraints and increase the complexity of placement problem. To overcome these challenges, we propose an actor-critic-based distributed application placement technique, working based on the IMPortance weighted Actor-Learner Architectures (IMPALA). IMPALA is known for efficient distributed experience trajectory generation that significantly reduces exploration costs of agents. Besides, it uses an adaptive off-policy correction method for faster convergence to optimal solutions. Our technique uses recurrent layers to capture temporal behaviors of input data and a replay buffer to improve the sample efficiency. The performance results, obtained from simulation and testbed experiments, demonstrate that our technique significantly improves execution cost of IoT applications up to 30% compared to its counterparts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Huang, L. P., Bi, S., & Zhang, Y. A. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,152
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner ArchitecturesPreprint
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01561 · Cited by 1,049
Fog Computing: A Taxonomy, Survey and Future Directions
Mahmud, R., Ramamohanarao, K., & Buyya, R. (2017) · Internet of things · doi:10.1007/978-981-10-5861-5_5 · Cited by 935
Self-citationSurvey on fog computing: architecture, key technologies, applications and open issues
Hu, P., Dhelim, S., Ning, H., & Qiu, T. (2017) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.09.002 · Cited by 880
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2876279 · Cited by 680
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues
Sun, Y., Peng, M., Zhou, Y., Huang, Y., & Mao, S. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2924243 · Cited by 678
List Scheduling Algorithm for Heterogeneous Systems by an Optimistic Cost Table
Arabnejad, H., & Barbosa, J. G. (2014) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2013.57 · Cited by 660
Learning-Based Computation Offloading for IoT Devices With Energy Harvesting
Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890685 · Cited by 560
QoS-Aware Deployment of IoT Applications Through the Fog
Antonio Brogi, A., & Forti, S. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2701408 · Cited by 486
Delay-Aware Microservice Coordination in Mobile Edge Computing: A Reinforcement Learning Approach
Wang, S., Guo, Y., Zhang, N., Yang, P., Zhou, A., & Shen, X. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2957804 · Cited by 477
Fog Computing in Healthcare Internet of Things: A Case Study on ECG Feature Extraction
Gia, T. N., Jiang, M., Rahmani, A.-M., Westerlund, T., Liljeberg, P., & Tenhunen, H. (2015) · doi:10.1109/cit/iucc/dasc/picom.2015.51 · Cited by 448
Fast Adaptive Task Offloading in Edge Computing Based on Meta Reinforcement Learning
Wang, J., Hu, J. W., Min, G., Zomaya, A. Y., & Georgalas, N. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3014896 · Cited by 426
Mobility-Aware Application Scheduling in Fog Computing
Bittencourt, L. F., Diaz‐Montes, J., Buyya, R., RANA, O. F., & Parashar, M. (2017) · IEEE Cloud Computing · doi:10.1109/mcc.2017.27 · Cited by 408
Self-citationRa, M.-R., Sheth, A. N., Mummert, L., Pillai, P. S., Wetherall, D. J., & Govindan, R. (2011) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/1999995.2000000 · Cited by 360
Goudarzi, M., Wu, H., Palaniswami, M. S., & Buyya, R. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.2967041 · Cited by 357
Self-citationEnergy-Efficient Resource Allocation for Mobile Edge Computing-Based Augmented Reality Applications
Al-Shuwaili, A., & Simeone, O. (2017) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2017.2696539 · Cited by 351
A computation offloading method over big data for IoT-enabled cloud-edge computing
Xu, X., Liu, Q., Luo, Y., Peng, K., Zhang, X., Meng, S., & Qi, L. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2018.12.055 · Cited by 336
Resource Allocation Based on Deep Reinforcement Learning in IoT Edge Computing
Xiong, X., Zheng, K., Lei, L., & Hou, L. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.2986615 · Cited by 301
Tuli, S., Ilager, S., Ramamohanarao, K., & Buyya, R. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3017079 · Cited by 276
Self-citationQi, Q., Wang, J., Ma, Z., Sun, H., Cao, Y., Zhang, L., & Liao, J. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2894437 · Cited by 268
Huang, L. P., Xu, F., Zhang, C., Qian, L., & Wu, Y. (2018) · Digital Communications and Networks · doi:10.1016/j.dcan.2018.10.003 · Cited by 265
Optimal Joint Scheduling and Cloud Offloading for Mobile Applications
Mahmoodi, S. E., Uma, R. N., & Subbalakshmi, K. P. (2016) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2016.2560808 · Cited by 236
iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks
Chen, J., Chen, S., Wang, Q., Cao, B., Feng, G., & Hu, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2913162 · Cited by 228
Lu, H., Gu, C., Luo, F., Ding, W., & Liu, X. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.07.019 · Cited by 211
Optimal Application Deployment in Resource Constrained Distributed Edges
Deng, S., Xiang, Z., Taheri, J., Khoshkholghi, M. A., Yin, J., Zomaya, A. Y., & Dustdar, S. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.2970698 · Cited by 199
Distributed Deep Learning-based Offloading for Mobile Edge Computing Networks
Huang, L. P., Xu, F., Feng, A., Huang, Y., & Qian, L. (2018) · Mobile Networks and Applications · doi:10.1007/s11036-018-1177-x · Cited by 188
Edge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things
Lu, H., He, X., Du, M., Ruan, X., Sun, Y., & Wang, K. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2981557 · Cited by 175
Gazori, P., Rahbari, D., & Nickray, M. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.09.060 · Cited by 130
An Efficient Application Partitioning Algorithm in Mobile Environments
Wu, H., Knottenbelt, W. J., & Wolter, K. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2891695 · Cited by 125
Qiu, X., Zhang, W., Chen, W., & Zheng, Z.-B. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3042599 · Cited by 93
Delay-Sensitive Multi-Period Computation Offloading with Reliability Guarantees in Fog Networks
Wang, J., Liu, K., Li, B., Liu, T., Li, R., & Han, Z. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2918773 · Cited by 91
A fast hybrid multi-site computation offloading for mobile cloud computing
Goudarzi, M., Zamani, M., & Haghighat, A. T. (2016) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2016.12.031 · Cited by 69
Self-citationLearning Everywhere: Pervasive Machine Learning for Effective High-Performance Computation
Fox, G., Adiga, A., Chen, J., Beckstein, O., Jha, S., Glazier, J. A., Kadupitiya, J., Jadhao, V., Kim, M., Qiu, J., et al. (2019) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2019.00081 · Cited by 56
Deng, Q., Goudarzi, M., & Buyya, R. (2021) · International Workshop on Big Data in Emergent Distributed Environments (BiDEDE) · doi:10.1145/3460866.3461768 · Cited by 51
Self-citationGoudarzi, M., Palaniswami, M. S., & Buyya, R. (2021) · Annals of Computer Science and Information Systems · doi:10.15439/2021f005 · Cited by 27
Self-citationA fog-driven dynamic resource allocation technique in ultra dense femtocell networks
Goudarzi, M., Palaniswami, M. S., & Buyya, R. (2019) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102407 · Cited by 25
Self-citationMobile Cloud Computing: A multisite computation offloading
Goudarzi, M., Movahedi, Z., & Nazari, M. (2016) · International Symposium on Telecommunications · doi:10.1109/istel.2016.7881904 · Cited by 15
Self-citation
Research works printed but not linked: 2
- Jeff, ML for system, system for ML, keynote talk in workshop on ML for systems, NIPS
- Appleyard, 2016, Optimizing performance of recurrent neural networks on GPUs. corr abs/1604.01946 (2016) · arXiv
Cited by
Sorted by citations
A review on TinyML: State-of-the-art and prospects
Ray, P. P. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.11.019 · Cited by 460
Walia, G. K., Kumar, M., & Gill, S. S. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3338015 · Cited by 299
Wang, Z., Goudarzi, M., Gong, M., & Buyya, R. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.10.012 · Cited by 209
Self-citationDuan, Q., Huang, J., Hu, S., Deng, R., Lu, Z., & Yu, S. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3316615 · Cited by 189
Jayanetti, A., Halgamuge, S. K., & Buyya, R. (2024) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2024.3360448 · Cited by 140
Self-citation
Metadata
- Authors
- Goudarzi, M., Palaniswami, M., & Buyya, R.
- Affiliations
- The University of Melbourne (AU)
- Venue
- IEEE Transactions on Mobile Computing
- Published
- 2023-05-01
- Volume
- 22
- Issue
- 5
- Pages
- 2491-2505
- ISSN
- 1536-1233 (ISSN-L)
- 1558-0660
- 2161-9875
- OpenAlex
- W3208229128
- Cited by
- 155
- References
- 40
- Citations by year
- 2026: 31 · 2025: 41 · 2024: 40 · 2023: 27
- FWCI
- 9.31
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Mobile Computing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- application placement, edge computing, fog computing, deep reinforcement learning, distributed training, impala, actor-critic, directed acyclic graphs, Internet of Things, task placement, execution cost, recurrent neural networks, replay buffer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
