The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments Using A3C Learning and Residual Recurrent Neural Networks
Open accessAbstract
The ubiquitous adoption of Internet-of-Things (IoT) based applications has resulted in the emergence of the Fog computing paradigm, which allows seamlessly harnessing both mobile-edge and cloud resources. Efficient scheduling of application tasks in such environments is challenging due to constrained resource capabilities, mobility factors in IoT, resource heterogeneity, network hierarchy, and stochastic behaviors. Existing heuristics and Reinforcement Learning based approaches lack generalizability and quick adaptability, thus failing to tackle this problem optimally. They are also unable to utilize the temporal workload patterns and are suitable only for centralized setups. However, asynchronous-advantage-actor-critic (A3C) learning is known to quickly adapt to dynamic scenarios with less data and residual recurrent neural network (R2N2) to quickly update model parameters. Thus, we propose an A3C based real-time scheduler for stochastic Edge-Cloud environments allowing decentralized learning, concurrently across multiple agents. We use the R2N2 architecture to capture a large number of host and task parameters together with temporal patterns to provide efficient scheduling decisions. The proposed model is adaptive and able to tune different hyper-parameters based on the application requirements. We explicate our choice of hyper-parameters through sensitivity analysis. The experiments conducted on real-world data set show a significant improvement in terms of energy consumption, response time, Service-Level-Agreement and running cost by 14.4, 7.74, 31.9, and 4.64 percent, respectively when compared to the state-of-the-art algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Internet of Things (IoT): A vision, architectural elements, and future directions
Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2013.01.010 · Cited by 12,293
Self-citation(2016) · Cited by 11,679
(2017) · Cited by 11,625
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Calheiros, R. N., Ranjan, R., Beloglazov, A., De Rose, C. A. F., & Buyya, R. (2011) · Software: Practice and Experience · doi:10.1002/spe.995 · Cited by 4,948
Self-citationNonlinear Multiobjective Optimization
Miettinen, K. (1998) · International Series in Operations Research & Management Science · doi:10.1007/978-1-4615-5563-6 · Cited by 4,710
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Nonlinear Multiobjective Optimization
Mardle, S., & Miettinen, K. M. (2000) · The Journal of the Operational Research Society · doi:10.2307/254267 · Cited by 3,049
Efficient multi-user computation offloading for mobile-edge cloud computing
Chen, X., Jiao, L., Li, W., & Fu, X. (2016) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2487344 · Cited by 2,675
Gate-variants of Gated Recurrent Unit (GRU) neural networks
Dey, R., & Salem, F. M. (2017) · 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS) · doi:10.1109/mwscas.2017.8053243 · Cited by 2,186
Beloglazov, A., & Buyya, R. (2012) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.1867 · Cited by 1,864
Self-citationGupta, H., Vahid Dastjerdi, A., Ghosh, S. K., & Buyya, R. (2017) · Software: Practice and Experience · doi:10.1002/spe.2509 · Cited by 1,649
Self-citationLearning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing
Li, H., Ota, K., & Dong, M. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700202 · Cited by 1,566
Wright, S. J. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0892-3 · Cited by 1,438
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353
A Survey of Fog Computing: Concepts, Applications and Issues
Yi, S., Li, C., & Li, Q. (2015) · Proceedings of the 2015 Workshop on Mobile Big Data · doi:10.1145/2757384.2757397 · Cited by 1,339
Rahmani, A. M., Gia, T. N., Negash, B., Anzanpour, A., Azimi, I., Jiang, M., & Liljeberg, P. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2017.02.014 · Cited by 1,198
Huang, L., Bi, S., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,040
Tuli, S., Basumatary, N., Gill, S. S., Kahani, M., Arya, R. C., Wander, G. S., & Buyya, R. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2019.10.043 · Cited by 669 · cited 2×
“Thus, To minimize the above mentioned metrics, as done in various prior works [16], [35], we define Loss i as a convex combination of these metrics for interval SI i (cid”
BackgroundMethodSelf-citationWireless energy harvesting for the Internet of Things
Kamalinejad, P., Mahapatra, C., Sheng, Z., Mirabbasi, S., M. Leung, V. C., & Guan, Y. L. (2015) · IEEE Communications Magazine · doi:10.1109/mcom.2015.7120024 · Cited by 664
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
Pathak, D., Krahenbuhl, P., & Darrell, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.209 · Cited by 617
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01561 · Cited by 553
QoS-Aware Deployment of IoT Applications Through the Fog
Brogi, A., & Forti, S. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2701408 · Cited by 472
Speedup versus efficiency in parallel systems
Eager, D. L., Zahorjan, J., & Lazowska, E. D. (1989) · IEEE Transactions on Computers · doi:10.1109/12.21127 · Cited by 437
FogBus: A Blockchain-based Lightweight Framework for Edge and Fog Computing
Tuli, S., Mahmud, R., Tuli, S., & Buyya, R. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.04.050 · Cited by 392
Self-citationOptimized IoT service placement in the fog
Skarlat, O., Nardelli, M., Schulte, S., Borkowski, M., & Leitner, P. (2017) · Service Oriented Computing and Applications · doi:10.1007/s11761-017-0219-8 · Cited by 356
Theoretical modelling of fog computing: a green computing paradigm to support IoT applications
Sarkar, S., & Misra, S. (2016) · IET Networks · doi:10.1049/iet-net.2015.0034 · Cited by 344
Quality of Experience (QoE)-aware placement of applications in Fog computing environments
Mahmud, R., Srirama, S. N., Ramamohanarao, K., & Buyya, R. (2019) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2018.03.004 · Cited by 313
Self-citationStatistical Characterization of Business-Critical Workloads Hosted in Cloud Datacenters
Shen, S., Van Beek, V., & Iosup, A. (2015) · 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing · doi:10.1109/ccgrid.2015.60 · Cited by 286
Deep Reinforcement Learning for Mobile 5G and Beyond: Fundamentals, Applications, and Challenges
Xiong, Z., Zhang, Y., Niyato, D., Deng, R., Wang, P., & Wang, L.-C. (2019) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2019.2903655 · Cited by 285
Cheng, M., Li, J., & Nazarian, S. (2018) · 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2018.8297294 · Cited by 196
Towards task scheduling in a cloud-fog computing system
Pham, X.-Q., & Huh, E.-N. (2016) · 2016 18th Asia-Pacific Network Operations and Management Symposium (APNOMS) · doi:10.1109/apnoms.2016.7737240 · Cited by 194
Prioritized task scheduling in fog computing
Choudhari, T., Moh, M., & Moh, T.-S. (2018) · doi:10.1145/3190645.3190699 · Cited by 168
Energy-Efficient Scheduling for Real-Time Systems Based on Deep Q-Learning Model
Zhang, Q., Lin, M., Yang, L. T., Chen, Z., & Li, P. (2019) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2017.2743704 · Cited by 166
Pham, X.-Q., Man, N. D., Tri, N. D. T., Thai, N. Q., & Huh, E.-N. (2017) · International Journal of Distributed Sensor Networks · doi:10.1177/1550147717742073 · Cited by 159
A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling
Zhang, Q., Lin, M., Yang, L. T., Chen, Z., Khan, S. U., & Li, P. (2019) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2018.2867482 · Cited by 158
Deep and reinforcement learning for automated task scheduling in large‐scale cloud computing systems
Rjoub, G., Bentahar, J., Wahab, O. A., & Bataineh, A. S. (2020) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.5919 · Cited by 156
A Survey on Energy Conserving Mechanisms for the Internet of Things: Wireless Networking Aspects
Abbas, Z., & Yoon, W. (2015) · Sensors · doi:10.3390/s151024818 · Cited by 126
Residual Recurrent Neural Networks for Learning Sequential Representations
Yue, B., Fu, J., & Liang, J. (2018) · Information · doi:10.3390/info9030056 · Cited by 114
Delay-Sensitive Multi-Period Computation Offloading with Reliability Guarantees in Fog Networks
Wang, J., Liu, K., Li, B., Liu, T., Li, R., & Han, Z. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2918773 · Cited by 94
Li, F., & Hu, B. (2019) · International Conference on Big Data and Computing (ICBDC) · doi:10.1145/3335484.3335513 · Cited by 87
Energy efficiency for cloud computing system based on predictive optimization
Bui, D.-M., Yoon, Y.-I., Huh, E., Jun, S.-I., & Lee, S. (2016) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2016.11.011 · Cited by 80
Large scale recurrent neural network on GPU
Li, B., Zhou, E., Huang, B., Duan, J., Wang, Y., Xu, N., Zhang, J., & Yang, H. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889433 · Cited by 76
Cheng, M., Li, J., & Nazarian, S. (2018) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.5555/3201607.3201635 · Cited by 74
Learn-as-you-go with Megh: Efficient Live Migration of Virtual Machines
Basu, D., Wang, X., Hong, Y., Chen, H., & Bressan, S. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2893648 · Cited by 74
Roselli, L., Mariotti, C., Mezzanotte, P., Alimenti, F., Orecchini, G., Virili, M., & Carvalho, N. B. (2015) · IEEE Topical Conference on Wireless Sensors and Sensor Networks (WiSNeT) · doi:10.1109/wisnet.2015.7127402 · Cited by 61
Convergence Problems of General-Sum Multiagent Reinforcement Learning
Bowling, M. (2000) · Cited by 60
Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computation
Fox, G., Adiga, A., Chen, J., Beckstein, O., Jha, S., Glazier, J. A., Kadupitiya, J., Jadhao, V., Kim, M., Qiu, J., et al. (2019) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2019.00081 · Cited by 56
Vehicular Edge Computing via Deep Reinforcement Learning
Qi, Q., & Ma, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.04290 · Cited by 17
Deep learning neural networks optimization using hardware cost penalty
Doshi, R., Hung, K.-W., Liang, L., & Chiu, K. H. (2016) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2016.7538957 · Cited by 9
Cited by
A Survey of Recent Advances in Edge-Computing-Powered Artificial Intelligence of Things
Chang, Z., Liu, S., Xiong, X., Cai, Z., & Tu, G. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3088875 · Cited by 418
Joint Task Offloading and Resource Allocation for Energy-Constrained Mobile Edge Computing
Jiang, H., Dai, X., Xiao, Z., & Iyengar, A. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3150432 · Cited by 411
Walia, G. K., Kumar, M., & Gill, S. S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3338015 · Cited by 279
AI-based fog and edge computing: A systematic review, taxonomy and future directions
Iftikhar, S., Gill, S. S., Song, C., Xu, M., Aslanpour, M. S., Toosi, A. N., Du, J., Wu, H., Ghosh, S., Chowdhury, D., et al. (2023) · Internet of Things · doi:10.1016/j.iot.2022.100674 · Cited by 260
Goudarzi, M., Palaniswami, M., & Buyya, R. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3123165 · Cited by 155
Self-citation
Metadata
- Authors
- Tuli, S., Ilager, S., Ramamohanarao, K., & Buyya, R.
- Venue
- IEEE Transactions on Mobile Computing
- Published
- 2022-03-01
- Volume
- 21
- Issue
- 3
- Pages
- 940-954
- ISSN
- 1536-1233 (ISSN-L)
- 1558-0660
- 2161-9875
- OpenAlex
- W3059506646
- Cited by
- 257
- References
- 54
- FWCI
- 16.27
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.