The DOI matches its Crossref record, with a small difference in title or year; 39 of the 40 references it cites were found in a registry or an index.
A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments
Open accessAbstract
Fog/Edge computing is a novel computing paradigm supporting resource-constrained Internet of Things (IoT) devices by placement of their tasks on edge and/or cloud servers. Recently, several Deep Reinforcement Learning (DRL)-based placement techniques have been proposed in fog/edge computing environments, which are only suitable for centralized setups. The training of well-performed DRL agents requires manifold training data while obtaining training data is costly. Hence, these centralized DRL-based techniques lack generalizability and quick adaptability, thus failing to efficiently tackle application placement problems. Moreover, many IoT applications are modeled as Directed Acyclic Graphs (DAGs) with diverse topologies. Satisfying dependencies of DAG-based IoT applications incur additional constraints and increase the complexity of placement problem. To overcome these challenges, we propose an actor-critic-based distributed application placement technique, working based on the IMPortance weighted Actor-Learner Architectures (IMPALA). IMPALA is known for efficient distributed experience trajectory generation that significantly reduces exploration costs of agents. Besides, it uses an adaptive off-policy correction method for faster convergence to optimal solutions. Our technique uses recurrent layers to capture temporal behaviors of input data and a replay buffer to improve the sample efficiency. The performance results, obtained from simulation and testbed experiments, demonstrate that our technique significantly improves execution cost of IoT applications up to 30% compared to its counterparts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Huang, L. P., Bi, S., & Zhang, Y. A. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,152
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner ArchitecturesPreprint
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01561 · Cited by 1,049
Fog Computing: A Taxonomy, Survey and Future Directions
Mahmud, R., Ramamohanarao, K., & Buyya, R. (2017) · Internet of things · doi:10.1007/978-981-10-5861-5_5 · Cited by 935
Self-citationSurvey on fog computing: architecture, key technologies, applications and open issues
Hu, P., Dhelim, S., Ning, H., & Qiu, T. (2017) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.09.002 · Cited by 880
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2876279 · Cited by 680
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues
Sun, Y., Peng, M., Zhou, Y., Huang, Y., & Mao, S. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2924243 · Cited by 678
List Scheduling Algorithm for Heterogeneous Systems by an Optimistic Cost Table
Arabnejad, H., & Barbosa, J. G. (2014) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2013.57 · Cited by 660
Learning-Based Computation Offloading for IoT Devices With Energy Harvesting
Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890685 · Cited by 560
Research works printed but not linked: 2
- Jeff, ML for system, system for ML, keynote talk in workshop on ML for systems, NIPS
- Appleyard, 2016, Optimizing performance of recurrent neural networks on GPUs. corr abs/1604.01946 (2016) · arXiv
Cited by
Sorted by citations
Items 101–160 of 160
Optimizing Service Placement in Fog Environments with Deep Reinforcement Learning
Liu, W., Xie, Q., Arain, T. A., & Ali, U. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-1581-3_18 · Cited by 1
Neural Adaptive Dependent Task Placement for IoT Streaming Applications
Ding, Y., Li, Y., Cao, C., & Liu, Q. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227758 · Cited by 1
Analysis of Learning-Based Methods for Intelligent Edge Computing
Sandar, N. M., & Win, T. A. (2026) · Journal of Electronics and Informatics · doi:10.36548/jei.2025.4.003 · Cited by 1
Guo, Y., Chang, S., Ji, D., Qin, S., & Xu, T. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.103695 · Cited by 1
Latency and trust constrained fog node selection using deep reinforcement learning
Babu, A., & Bala, G. J. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1723498 · Cited by 1
Zhao, Y., Wang, C., Yan, C., Li, X., & Jiang, C. (2026) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2026.3656552 · Cited by 1
Distributed Task Scheduling in Collaborative Edge Infrastructures with Graph Reinforcement Learning
Kokkinakis, P., Soumplis, P., & Varvarigos, E. (2025) · Globecom · doi:10.1109/globecom59602.2025.11432010 · Cited by 1
Priority-Aware Task Offloading and Scheduling for Dynamic Task Management in Internet of Things
Pawar, V. N., & Mini, S. (2026) · Wireless Personal Communications · doi:10.1007/s11277-026-12138-6 · Cited by 1
A Comprehensive Review of Tiny Machine Learning: Enabling AI on Resource-Constrained Devices
ABBAS, N. A. B., & Ahmad, M. R. (2025) · ELEKTRIKA- Journal of Electrical Engineering · doi:10.11113/elektrika.v24n2.555 · Cited by 1
Verma, N. K., & Naik, K. J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132044 · Cited by 1
Distributed Learning for Application Placement at the Edge Minimizing Active Nodes
Torres-Pérez, C., Coronado, E., Cervelló-Pastor, C., Camargo, J. S., & Siddiqui, M. S. (2023) · International Conference on 6G Networking (6GNet) · doi:10.1109/6gnet58894.2023.10317692 · Cited by 1
Singh, P. P., & Saini, K. S. (2024) · International Conference on Sentiment Analysis and Deep Learning (ICSADL) · doi:10.1109/icsadl61749.2024.00075 · Cited by 1
Rezazadeh, A., Abednezhad, D., Jones, D., & Lutfiyya, H. L. (2024) · Conference on Cloud and Internet of Things (CIoT) · doi:10.1109/ciot63799.2024.10757095 · Cited by 1
Energy-Optimized Offloading of Delay-Sensitive Tasks in Hybrid Edge-Cloud Computing
Yuan, H., Wang, S., Ma, Y., Bi, J., Yang, J., Zhang, J., & Zhou, M. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831549 · Cited by 1
Supervised Learning for Optimal Latency-Aware Microservice Placement in Edge Computing Environments
Kasahara, H., Kitajima, R., Towada, J., & Sato, O. (2025) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms57970.2025.11073720 · Cited by 1
Bhargavi, K., & Shiva, S. G. (2025) · Mathematics · doi:10.3390/math13172768 · Cited by 1
Utilizing Technology and Management of Fog Computing
Tewari, S., Tewari, N., & Joshi, M. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-2538-2_49 · Cited by 1
DQN-based Intelligent Application Placement with Delay-Priority in Multi MEC Systems
Camargo, J. S., Coronado, E., Torres-Pérez, C., Palomares, J., & Siddiqui, M. S. (2023) · Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) · doi:10.1109/eucnc/6gsummit58263.2023.10188300 · Cited by 1
Quality of Service-Based Resource Management in Fog Computing
Jain, V., & Kumar, B. (2022) · International Journal of Cloud Applications and Computing · doi:10.4018/ijcac.309934
MOTIFSM: Cloudera Motif DNA Finding Algorithm
Allam, T. M. (2023) · International Journal of Information Technology and Computer Science · doi:10.5815/ijitcs.2023.04.02
Panek, G., Nouar, N. E.-H., Fajjari, I., Matysiak, P., & Tarasiuk, H. (2023) · Globecom · doi:10.1109/globecom54140.2023.10437065
Service Management in Dynamic Edge Environments
Torres-Pérez, C., Coronado, E., Cervelló-Pastor, C., & Siddiqui, M. S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-48803-0_38
Muhamad, A., & Hussin, M. B. (2024) · IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS) · doi:10.1109/i2cacis61270.2024.10649846
Akhter, N., Jin, J., Mahmud, R., But, J., Gao, L., & Xiang, Y. (2024) · IEEE Annual Congress on Artificial Intelligence of Things (AIoT) · doi:10.1109/aiot63253.2024.00044
Duplicate Data Elimination Optimization Strategy Based on Edge Service Management
Pu, Y., Xu, X., Qi, L., Dou, W., Yu, J., & Li, Z. (2024) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/cyberscitech64112.2024.00083
Torres-Pérez, C., Coronado, E., Cervelló-Pastor, C., Palomares, J., Carmona-Cejudo, E., & Siddiqui, M. S. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.111008
Algorithmic Fault Impact Evaluation in Mobile Computation Offloading
Pirbasti, M. R., & Das, O. (2024) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc63386.2024.00058
Yao, Y., Yao, S., Zhou, N., Liang, X., Yang, C., & Gui, W. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4503-9_7
Improved PPO-Based Task Offloading Strategies for Smart Grids
Wang, Q., & Zhou, Y. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.065465
Qian, L., Huang, Y., Cheng, C., & Xu, H. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9958-2_25
Masoud, M., & Ganti, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06326-7_7
Collaborative Offloading for Interacting Users in Cloud–Edge–Terminal Networks
Yan, X., Chen, N., Ni, Z., Xue, H., Sun, H., & Chen, H. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3625670
Patro, S., Swain, S. K., & Mangalampalli, S. S. (2025) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.70299
Towards Efficient Allocation of Tasks in Dag-Based Workflows Across Federated Fog Systems
Gupta, G., Gupta, A., Salehi, M. A., Anta, A. F., Bhowmick, S., Aguilar, J., & Kourouma, M. K. (2025) · IEEE Cloud Summit · doi:10.1109/cloud-summit64795.2025.00026
REPA: Resilient and Proactive Resource Allocation for Last-Mile Heterogeneous Edge-IoT Systems
AlQerm, I., Cheng, N., & Pan, J. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3629135
Zheng, Y., Zhu, R., Fang, J., Lu, Y., & Xiang, W. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3656573
Microservice provisioning and event-driven adaptation in heterogeneous IoT settings
Li, Y., Liu, F., Chang, T.-Y., Hsu, C.-H., & Venkatasubramanian, N. (2026) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2026.102195
Li, X. (2025) · International Conference on Computer Communication, Networks and Information Science (CCNIS) · doi:10.1109/ccnis69465.2025.00065
Deep Reinforcement Learning for Integrated Order Picking and Inventory Control
Devabhaktuni, R. (2025) · International Conference on Engineering and Emerging Technologies (ICEET) · doi:10.1109/iceet67911.2025.11424113
Gandhi, R. A., & Singla, J. (2026) · International Conference on Image Processing and Capsule Networks (ICIPCN) · doi:10.1109/icipcn67432.2026.11438715
Vardar, M., & Yadav, S. K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-18144-2_19
Multi‐Objective Deep Q‐Network for Efficient Application Placement in Fog Computing
Bochra, A., & Amraoui, A. (2026) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70695
He, Z., Zhai, X., Zhao, M., Zhou, W., & Li, K. (2026) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2026.3686168
Nugraha, A., Shidik, G. F., Suhartanto, H. S., & Marjuni, A. (2026) · International Journal of Cognitive Computing in Engineering · doi:10.1016/j.ijcce.2026.04.004
Yadav, P. P., & Reddy, T. B. (2026) · IEEE International Conference on Emerging Computing and Intelligent Technologies (ICoECIT) · doi:10.1109/icoecit68303.2026.11498047
Kumari, S., Jena, A. K., Behera, D. K., & Panda, A. R. (2026) · International Conference on Computational Intelligence and Networks (CINE) · doi:10.1109/cine68769.2026.11502660
Intent-Driven CPU Heterogeneity Management in the Cloud-Edge ContinuumPreprint
Dib, D., & Metsch, T. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6832338
CANOPY: Alocação Orientada por Predição Energética de Aplicações no Contínuo Computacional
Reis, G., Figueiredo, G. B., Santos, B. P., Prazeres, C. V. S., Veiga, L., & Peixoto, M. L. M. (2026) · Anais do Workshop de Computação Urbana (CoUrb) · doi:10.5753/courb.2026.23223
Khaleel, M. I. (2026) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2026.103311
M, G. N., & Mahesh, G. (2026) · ITM Web of Conferences · doi:10.1051/itmconf/20268701019
Scale-Reconfigurable Multi-Agent Reinforcement Learning for Aerial Networks in Dynamic Environments
Kim, G. S., Roh, E. J., Jung, S., Park, S., & Kim, J. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3709180
Multi-Agent Cloud-Edge Offloading Scheduling for Heterogeneous DAGs with Deadlines
Wang, J., Wang, J., Gao, Y., & Lakhan, A. R. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582669
A hierarchical fog computing framework for efficient and scalable task offloading in IoT systems
Khan, S., Loh, W.-K., & Babar, M. (2026) · Array · doi:10.1016/j.array.2026.101187
Resource Coordination and Joint Optimization in Cloud-Edge-End Systems
(2026) · Frontiers research topics · doi:10.3389/978-2-8325-8469-9
Resource Allocation in Multi-Fog/Cloud Systems Using Reinforcement Learning
Mokhtari, M., & Ganti, S. (2025) · IEEE International Conference on Big Data and Cloud Computing (BDCloud) · doi:10.1109/bdcloud67918.2025.00030
Multi-UAV Path Planning in 3D based on APF-MADDPGPreprint
Bai, M., Chen, Y., & Liu, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-6267765/v1
Xiao, J., Zhang, Z., & Zhou, Y. (2026) · Computer Networks · doi:10.1016/j.comnet.2026.112702
Kasahara, H., Kitajima, R., Towada, J., & Sato, O. (2026) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms69089.2026.11668339
Bouafia, Z., Mededjel, M., Ghalem, B., & Mokadem, R. (2026) · Cluster Computing · doi:10.1007/s10586-026-06601-z
Zuo, L., He, D., Zhang, K., & Wang, Y. (2026) · Cluster Computing · doi:10.1007/s10586-026-06682-w
Metadata
- Authors
- Goudarzi, M., Palaniswami, M., & Buyya, R.
- Affiliations
- The University of Melbourne (AU)
- Venue
- IEEE Transactions on Mobile Computing
- Published
- 2023-05-01
- Volume
- 22
- Issue
- 5
- Pages
- 2491-2505
- ISSN
- 1536-1233 (ISSN-L)
- 1558-0660
- 2161-9875
- OpenAlex
- W3208229128
- Cited by
- 155
- References
- 40
- Citations by year
- 2026: 31 · 2025: 41 · 2024: 40 · 2023: 27
- FWCI
- 9.31
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Mobile Computing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- application placement, edge computing, fog computing, deep reinforcement learning, distributed training, impala, actor-critic, directed acyclic graphs, Internet of Things, task placement, execution cost, recurrent neural networks, replay buffer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
