Bandit Convex Optimization for Scalable and Dynamic IoT Management
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
This paper deals with online convex optimization involving both time-varying loss functions, and time-varying constraints. The loss functions are not fully accessible to the learner, and instead only the function values (also known as bandit feedback) are revealed at queried points. The constraints are revealed after making decisions, and can be instantaneously violated, yet they must be satisfied in the long term. This setting fits nicely the emerging online network tasks such as fog computing in the Internet-of-Things, where online decisions must flexibly adapt to the changing user preferences (loss functions), and the temporally unpredictable availability of resources (constraints). Tailored for such human-in-the-loop systems where the loss functions are hard to model, a family of online bandit saddle-point (BanSaP) schemes are developed, which adaptively adjust the online operations based on (possibly multiple) bandit feedback of the loss functions, and the changing environment. Performance here is assessed by: 1) dynamic regret that generalizes the widely used static regret and 2) fit that captures the accumulated amount of constraint violations. Specifically, BanSaP is proved to simultaneously yield sublinear dynamic regret and fit, provided that the best dynamic solutions vary slowly over time. Numerical tests in fog computation offloading tasks corroborate that our proposed BanSaP approach offers competitive performance relative to existing approaches that are based on gradient feedback.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“At this background, large state and action spaces in the whole offloading problem make the conventional learning algorithms [245] [256] [247] infeasible actually.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [247]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2010) · Cited by 8,261
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · cited 3×
“The online computational offloading task of fog computing in IoT [5], [6], [8] takes the form of BCO with long-term constraints (1).”
BackgroundMobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258 · cited 3×
“, N} with moderate processing capability; thus, part of workloads will be collaboratively processed by the local fog servers to meet the stringent latency requirement, and the rest will be offloaded to the remote data center in the cloud layer [6] (see Fig.”
BackgroundEfficient multi-user computation offloading for mobile-edge cloud computing
Chen, X., Jiao, L., Li, W., & Fu, X. (2016) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2487344 · Cited by 2,675 · cited 1×
“Online optimization with time-varying and long-term constraints is well motivated for applications from power control in wireless communication [26], geographical load balancing in cloud networks [14], [27], to computation offloading in fog computing [28], [29].”
BackgroundWeik, M. H. (2000) · Computer Science and Communications Dictionary · doi:10.1007/1-4020-0613-6_12436 · Cited by 2,570
Fog and IoT: An Overview of Research Opportunities
Chiang, M., & Zhang, T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2584538 · Cited by 2,362 · cited 2×
“Consider a mobile network with a sensor layer, a fog layer, and a cloud layer [3], [4].”
BackgroundStochastic Network Optimization with Application to Communication and Queueing Systems
Neely, M. J. (2010) · Synthesis lectures on communication networks · doi:10.2200/s00271ed1v01y201006cnt007 · Cited by 2,218
(2003) · Cited by 1,733
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S. (2012) · now publishers, Inc. eBooks · doi:10.1561/9781601986276 · Cited by 1,404
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S., & Cesa-Bianchi, N. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000024 · Cited by 1,295
Research works printed but not linked: 2
- chen, 2017, Bandit convex optimization for scalable and dynamic IoT management, arXiv preprint 1707 09060
- 2017, 8 Sensors to Help You Create a Smart Home
Cited by
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“At this background, large state and action spaces in the whole offloading problem make the conventional learning algorithms [245], [256], [247] infeasible actually.”
BackgroundIntroduction to Multi-Armed Bandits
Slivkins, A. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000068 · Cited by 780
Feng, J., Richard Yu, F., Pei, Q., Chu, X., Du, J., & Zhu, L. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2961707 · Cited by 332
“3) Dynamic Optimization: Moreover, most of the existing works [6], [10], [11] in the computation offloading decision and resource allocation strategies are optimized based on a one-time slot, and the long-term computation offloading performance cannot be characterized [12], [13].”
BackgroundLiu, S., Chen, P.-Y., Kailkhura, B., Zhang, G., Hero, A. O., & Varshney, P. K. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.3003837 · Cited by 185
“Online sensor management provides an example of such a scenario [27], [72].”
BackgroundDistributed Online Convex Optimization With Time-Varying Coupled Inequality Constraints
Yi, X., Li, X., Xie, L., & Johansson, K. H. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2964200 · Cited by 167
“The papers [13], [14] used a virtual queue, which essentially is a modified Lagrange multiplier, to handle stochastic and time-varying constraints and the authors of [15] extended the algorithm proposed in [14] with bandit feedback.”
MethodOnline Optimization as a Feedback Controller: Stability and Tracking
Colombino, M., Dall’Anese, E., & Bernstein, A. (2019) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2019.2906916 · Cited by 149
“, [15], [16] and pertinent references therein), feedback-based optimization algorithms exploits measurements form the system to steer the dynamical systems to the solution of an optimization problem with limited knowledge of the input-output map and no information about the disturbance.”
BackgroundDistributed Bandit Online Convex Optimization With Time-Varying Coupled Inequality Constraints
Yi, X., Li, X., Yang, T., Xie, L., Chai, T., & Johansson, K. H. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3030883 · Cited by 113
“An advantage of our algorithm is that the total number of rounds is not used in the algorithm, which is an improvement compared to the one-point sampling algorithms in [24], [26]–[29], [39], [41], although these paper did not consider bandit feedback for the time-varying inequality constraints or did not even consider…”
BackgroundMethodResultInfluentialOnline Primal-Dual Methods With Measurement Feedback for Time-Varying Convex Optimization
Bernstein, A., Dall’Anese, E., & Simonetto, A. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2896112 · Cited by 111
“In terms of existing literature on regret analysis for online dual and primal-dual gradient methods [19]–[23], the contributions consist in: (i) proving dynamic (as opposed to static) regret bounds; (ii) considering a general class of constrained optimization problems with feedback; (iii) assuming time-varying”
BackgroundClassification of optimization problems in fog computing
Bellendorf, J., & Mann, Z. Á. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.01.036 · Cited by 113
Learning and Management for Internet of Things: Accounting for Adaptivity and Scalability
Chen, T., Barbarossa, S., Wang, X., Giannakis, G. B., & Zhang, Z.-L. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2896243 · Cited by 106
Self-citationOptimization and Learning With Information Streams: Time-varying algorithms and applications
Dall’Anese, E., Simonetto, A., Becker, S., & Madden, L. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2968813 · Cited by 86
“Dynamic regret The dynamic regret is defined as [15], [16], [18], [19]:”
BackgroundMethodDistributed Online Optimization With Long-Term Constraints
Yuan, D., Proutière, A., & Shi, G. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3057601 · Cited by 83
Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning
Chen, T., Zhang, K., Giannakis, G. B., & Basar, T. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2021.3078100 · Cited by 67
“However, in many applications, including cloud-edge AI systems [17], autonomous driving [5], and other applications in IoT [18], the communication is costly and the latency caused by frequent communication becomes the bottleneck of the overall performance.”
BackgroundSelf-citationBLOT: Bandit Learning-Based Offloading of Tasks in Fog-Enabled Networks
Zhu, Z., Liu, T., Yang, Y., & Luo, X. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2927978 · Cited by 71
“Different from the model-based task offloading solutions proposed in [9], [11], [13], [14], [15], we do not require any specific relationships between the costs and the circuits in AS-1 andAS-3 as in [18].”
BackgroundMethodEnergy and task completion time trade-off for task offloading in fog-enabled IoT networks
Shahryari, O., Pedram, H., Khajehvand, V., & Fooladi, M. D. T. (2021) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2021.101395 · Cited by 74
Li, J., Gu, C., Wu, Z., & Huang, T. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2990796 · Cited by 60
Li, R., Zhou, Z., Chen, X., & Ling, Q. H. (2019) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2019.2937928 · Cited by 60
Hierarchical Energy-Efficient Mobile-Edge Computing in IoT Networks
Wang, Q., Tan, L. T., Hu, R. Q., & Qian, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3000193 · Cited by 65
“By doing so, both energy consumption and delay performance can be improved [5]–[9].”
BackgroundOnline Optimization for Real-Time Peer-to-Peer Electricity Market Mechanisms
Guo, Z., Pinson, P., Chen, S., Yang, Q., & Yang, Z. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3075707 · Cited by 63
“Despite many large-scale applications of online optimization, such as network resource allocation [29], [30], demand response [31], [32], and energy management [33], our work is the first application for P2P electricity market.”
BackgroundAn Asynchronous Online Negotiation Mechanism for Real-Time Peer-to-Peer Electricity Markets
Guo, Z., Pinson, P., Wu, Q., Chen, S., Yang, Q., & Yang, Z. (2021) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2021.3111869 · Cited by 58
“Despite many large-scale applications of online optimization, such as network resource allocation [24], [25], demand response [26], [27], and energy management [28], our work is the first application for P2P electricity markets.”
BackgroundSecure Mobile Edge Computing in IoT via Collaborative Online Learning
Li, B., Chen, T., & Giannakis, G. B. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2949504 · Cited by 49
“As cloud computing alone cannot handle such IoT requirements, edge computing has emerged as a promising complement to subside the computational resources from the cloud to edge servers [31], which also facilitates real-time computing [8], [9], [11], [27], [35].”
BackgroundSelf-citationYuan, X., Tian, H., Wang, H., Su, H., Liu, J., & Taherkordi, A. (2020) · IEEE Access · doi:10.1109/access.2020.2992639 · Cited by 49
“Without considering the priority of tasks could result in too many high priority data obtain insufficient resources [36].”
BackgroundYi, X., Li, X., Yang, T., Xie, L., Chai, T., & Johansson, K. H. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3230766 · Cited by 44
“Noting that the problem considered in this paper incorporates the problems considered in [25]–[31], [46]–[48], the examples studied in these papers, such as online job scheduling [26], online network resource allocation [28], mobile fog computing in IoT [29], online linear re-gressions [47], and online spam filtering t…”
BackgroundAdversarial Bandits with Knapsacks
Immorlica, N., Sankararaman, K. A., Schapire, R. E., & Slivkins, A. (2019) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2019.00022 · Cited by 88
Constrained Online Convex Optimization With Feedback Delays
Cao, X., Zhang, J., & Poor, H. V. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3030743 · Cited by 42
Energy-Efficient and delay-guaranteed computation offloading for fog-based IoT networks
Shahryari, O., Pedram, H., Khajehvand, V., & Fooladi, M. D. T. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107511 · Cited by 43
Online Convex Optimization of Multi-Energy Building-to-Grid Ancillary Services
Lesage‐Landry, A., Wang, H., Shames, I., Mancarella, P., & Taylor, J. A. (2019) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2019.2944328 · Cited by 35
Communication-Efficient Distributed Reinforcement LearningPreprint
Chen, T., Zhang, K., Giannakis, G. B., & Başar, T. (2018) · arXiv (Cornell University) · Cited by 35
Self-citationAn Online Learning Algorithm for Distributed Task Offloading in Multi-Access Edge Computing
Sun, Z., & Nakhai, M. R. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2991383 · Cited by 34
“Authors in [10] propose a gradient-free (bandit) algorithm to solve the computation offloading in IoT management, where the gradient of loss function can be estimated by revealed multiple loss function values according to the corresponding perturbed actions made by the device.”
BackgroundMethodProvisioning Edge Inference as a Service via Online Learning
Jin, Y., Jiao, L., Qian, Z., Zhang, S., Chen, N., Lu, S., & Wang, X. (2020) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/secon48991.2020.9158425 · Cited by 37
“Intuitively, in every time slot, one can “learn” in an online manner [10, 14] from the “penalty” incurred by the online decisions just made regarding the machine learning model provisioning and inference workload distribution after the inference workload actually arrives, and seek to make better decisions as time goes…”
BackgroundOptimisation methods for dispatch and control of energy storage with renewable integration
Guo, Z., Wei, W., Shahidehpour, M., Wang, Z., & Mei, S. (2022) · IET Smart Grid · doi:10.1049/stg2.12063 · Cited by 34
Bayesian Optimization for Online Management in Dynamic Mobile Edge Computing
Yan, J., Lu, Q., & Giannakis, G. B. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3307875 · Cited by 36
“…kernel hyperparameters on the performance of the proposed BO approaches, a 2-BS MEC system with M = 2 WDs is first considered, where the distances from the WDs to BSs are [20 , 13 , 15 , 18] meters, the Rician factor in (36) used to generate the channel gain is K = 4 , and η in (38) is set to 0 .”
BackgroundMethodSelf-citationChen, Y., Bernstein, A., Devraj, A. M., & Meyn, S. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147814 · Cited by 33
“These algorithms typically have high variance of the estimate if the evaluation is performed using one or two function evaluations [19].”
Distributed User Association With Grouping in Satellite–Terrestrial Integrated Networks
Dai, C., Li, S., Wu, J., & Chen, Q. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3122939 · Cited by 29
Optimal Distributed Convex Optimization on Slowly Time-Varying Graphs
Rogozin, A., Uribe, C. A., Gasnikov, A. V., Malkovsky, N., & Nedic, A. (2019) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2019.2949439 · Cited by 29
“These changes can come both from changing cost function and changing constraints [15], [16].”
BackgroundPersonalized optimization with user’s feedback
Simonetto, A., Dall’Anese, E., Monteil, J., & Bernstein, A. (2021) · Automatica · doi:10.1016/j.automatica.2021.109767 · Cited by 29
“, [36]–[38] and references therein); this is one of the strength of Bayesian modelling, which however comes at the cost of an increase in computational complexity due to the GP updates.”
BackgroundEnergy-Efficient Robust Computation Offloading for Fog-IoT Systems
Wu, Z., Li, B., Fei, Z., Zheng, Z., Li, B., & Han, Z. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2975056 · Cited by 31
Li, J., Li, C., Fan, J., & Huang, T. (2023) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2023.3327183 · Cited by 31
“Chen and Giannakis [33] introduced a saddle-point bandit method to solve OCO under time-changing inequality constraints.”
MethodDistributed Online Bandit Learning in Dynamic Environments Over Unbalanced Digraphs
Li, J., Li, C., Yu, W., Zhu, X., & Yu, X. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3093536 · Cited by 27
Distributed bandit online optimisation for energy management in smart grids
Zhao, Z., Xia, L., Jiang, L., Ge, Q., & Fang, Y. (2023) · International Journal of Systems Science · doi:10.1080/00207721.2023.2250043 · Cited by 25
“Bandit feedback is suitable for the situation where complete function or gradient information is not available and has gained widespread attention in recent years (Cao & Baar, 2022; Chen & Giannakis, 2019; Li et al., 2020; Xiong et al., 2023).”
BackgroundLi, J., Zhu, X., Wu, Z., & Huang, T. (2021) · Information Sciences · doi:10.1016/j.ins.2021.10.003 · Cited by 21
Predictive online convex optimization
Lesage‐Landry, A., Shames, I., & Taylor, J. A. (2019) · Automatica · doi:10.1016/j.automatica.2019.108771 · Cited by 21
Online Primal-Dual Mirror Descent under Stochastic Constraints
Wei, X., Yu, H., & Neely, M. J. (2020) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3392157 · Cited by 21
“…stochastic constraints, where g ti ( µ ) = g i ( µ, γ t ) and { γ t } T − 1 t =0 are i.i.d., is considered in the works such as (Yu et al., 2017; Chen and Giannakis, 2019; Liakopoulos et al., 2019), where a primal-dual proximal gradient algorithm is proposed and O ( √ T ) expected regret and…”
BackgroundInexact Online Proximal-gradient Method for Time-varying Convex Optimization
Ajalloeian, A., Simonetto, A., & Dall’Anese, E. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147467 · Cited by 21
“Inexactness of (3) is due to projections onto a restriction of Xk [20] or when (3) is not solved to convergence.”
BackgroundMethodInfluentialAdversarial Bandits with Knapsacks
Immorlica, N., Sankararaman, K. A., Schapire, R. E., & Slivkins, A. (2022) · Journal of the ACM · doi:10.1145/3557045 · Cited by 91
Online Optimization Using Zeroth Order Oracles
Shames, I., Selvaratnam, D., & Manton, J. H. (2019) · IEEE Control Systems Letters · doi:10.1109/lcsys.2019.2921593 · Cited by 20
“This corresponds to practical scenarios in which derivatives are either unavailable [6], [7], or the cost of computing them is prohibitive.”
BackgroundBandit Convex Optimization in Non-stationary EnvironmentsPreprint
Zhao, P., Wang, G., Zhang, L., & Zhou, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12340 · Cited by 19
“Recently, there are some studies on the worst-case dynamic regret of BCO problems (Yang et al., 2016; Chen and Giannakis, 2019).”
BackgroundDistributed Online Stochastic-Constrained Convex Optimization With Bandit Feedback
Wang, C., Xu, S., & Yuan, D. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3177644 · Cited by 18
A Primer on Zeroth-Order Optimization in Signal Processing and Machine LearningPreprint
Liu, S., Chen, P., Kailkhura, B., Zhang, G., Hero, A. O., & Varshney, P. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06224 · Cited by 18
“Online sensor management provides an example of such a scenario [27], [72].”
BackgroundMin-Max Optimization without Gradients: Convergence and Applications to Adversarial MLPreprint
Liu, S., Lu, S., Chen, X., Feng, Y., Xu, K., Al-Dujaili, A., Hong, M., & O’Reilly, U.-M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.13806 · Cited by 23
Online Primal-Dual Mirror Descent under Stochastic Constraints
Wei, X., Yu, H., & Neely, M. J. (2020) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/3393691.3394209 · Cited by 15
Liu, Q., Wu, W., Huang, L., & Fang, Z. (2021) · Performance Evaluation · doi:10.1016/j.peva.2021.102240 · Cited by 15
“2 Assumptions After specifying the problem, we introduce some assumptions in this paper, which are also common in the literature of constraint OCO [3, 4, 26].”
BackgroundMethodInfluentialLearn and Pick Right Nodes to Offload
Zhu, Z., Liu, T., Jin, S., & Luo, X. (2018) · Globecom · doi:10.1109/glocom.2018.8648042 · Cited by 14
Learning-NUM: Network Utility Maximization With Unknown Utility Functions and Queueing Delay
Fu, X., & Modiano, E. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3182890 · Cited by 14
Resource management in fog computing using greedy and semi-greedy spider monkey optimization
Hajam, S. S., & Sofi, S. A. (2023) · Soft Computing · doi:10.1007/s00500-023-09123-7 · Cited by 13
“These applications generally demand low latency and high computational capability which poses challenges to IoT devices as they are limited in computational capabilities, storage and energy Chen and Giannakis (2018).”
BackgroundPrimal and Dual Prediction-Correction Methods for Time-Varying Convex OptimizationPreprint
Bastianello, N., Simonetto, A., & Carli, R. (2020) · arXiv (Cornell University) · Cited by 13
Fu, X., & Modiano, E. (2021) · ACM International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing (MobiHoc) · doi:10.1145/3466772.3467031 · Cited by 12
“As the objective function of P is unknown, the problem has similar flavor to online/zerothorder optimization [20], [23], [24].”
BackgroundVisionScaling: Dynamic Deep Learning Model and Resource Scaling in Mobile Vision Applications
Choi, P., Ham, D., Kim, Y., & Kwak, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3349512 · Cited by 16
“For example, Chen and Giannakis [21] formulated an edge computing-assisted IoT (Internet of Things) device management problem as an OCO framework, and they assumed a time-varying cost and constraint functions are not observable before making decisions on control parameters.”
Secure Edge Computing in IoT via Online Learning
Li, B., Chen, T., Wang, X., & Giannakis, G. B. (2018) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2018.8645223 · Cited by 11
“To this end, edge computing has emerged as a promising solution by subsiding the computing capability from cloud to edge servers to facilitate real-time computation services [1], [2].”
BackgroundSelf-citationWang, J., & Hu, G. (2022) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6048 · Cited by 15
No-regret learning for repeated non-cooperative games with lossy bandits
Liu, W., Lei, J., Yi, P., & Hong, Y. (2023) · Automatica · doi:10.1016/j.automatica.2023.111455 · Cited by 10
Optimal Routing to Parallel Servers With Unknown Utilities—Multi-Armed Bandit With Queues
Fu, X., & Modiano, E. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3227136 · Cited by 10
“√ As such a Θ( T ) -regret can already be achieved by existing techniques [12], [13], we do not consider the instance-independent regret in this paper.”
BackgroundHierarchical Collaborative Cloud and Fog Computing in IoT Networks
Wang, Q., Tan, L. T., Hu, R. Q., & Wu, G. (2018) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp.2018.8555866 · Cited by 12
“The online bandit saddle-point (BanSaP) method [10], [11] is used to make the offloading decision variables, i.”
BackgroundMethodInfluentialOnline Primal-Dual Mirror Descent under Stochastic Constraints
Wei, X., Yu, H., & Neely, M. J. (2020) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3410048.3410051 · Cited by 10
Time-Varying Optimization of Networked Systems With Human Preferences
Ospina, A. M., Simonetto, A., & Dall’Anese, E. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3203467 · Cited by 10
“, [19] and [34]) consider the modified definition [∑T t=1 C t (y (x)) ]+ , which”
BackgroundMethodInfluentialOnline Network Utility Maximization: Algorithm, Competitive Analysis, and Applications
Cao, Y., Sun, B., & Tsang, D. H. K. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3199221 · Cited by 10
“Bandit feedback in online resource allocation has also attracted attention in recent years [14], [15], where only the function value of the action taken will be observed.”
BackgroundOnline Reconfiguration of IoT Applications in the Fog: The Information-Coordination Trade-Off
Donassolo, B., Legrand, A., Mertikopoulos, P., & Fajjari, I. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3097281 · Cited by 9
“In this context, the authors in [20] uses the Bandit Saddle-Point (BanSaP) algorithm with partial feedback to study the offload of tasks in a Fog environment.”
MethodHarnessing Bandit Online Learning to Low-Latency Fog Computing
Chen, T., & Giannakis, G. B. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461641 · Cited by 8
Self-citationJoint Learning and Control in Stochastic Queueing Networks with Unknown Utilities
Fu, X., & Modiano, E. (2022) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3570619 · Cited by 11
“We note that the optimal control problem can be subsumed into the general framework of network utility maximization (with unknown utility functions) [8, 23, 24] or reinforcement learning [12–14].”
BackgroundBandit Learning-Based Distributed Computation in Fog Computing Networks: A Survey
Tran‐Dang, H., Kwon, G.-H., & Kim, D. (2023) · IEEE Access · doi:10.1109/access.2023.3314889 · Cited by 9
“Based on this investigation, the studies [53] proposes amethod for managing the task offloading problems in the large-scale and dynamic IoT systems using BCO.”
MethodDistributed Management of Fluctuating Energy Resources in Dynamic Networked Systems
Cheng, X., Tsetis, I., & Maghsudi, S. (2024) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2024.3484149 · Cited by 8
“We compare our proposed algorithms with one benchmark, i.e., an adapted version of BanSaP algorithm proposed in [46].”
Multi-agent Online Optimization
Yuan, D., Proutière, A., & Shi, G. (2024) · Foundations and Trends® in Optimization · doi:10.1561/2400000037 · Cited by 18
Tracking and Regret Bounds for Online Zeroth-Order Euclidean and Riemannian Optimization
Maass, A. I., Manzie, C. G., Nešić, D., Manton, J. H., & Shames, I. (2022) · SIAM Journal on Optimization · doi:10.1137/21m1405551 · Cited by 8
“[41, 42] for machine learning, [35] for online controller tuning, [17] for deep neural networks, and [20] for mobile fog computing.”
BackgroundLiu, Q., Wu, W., Huang, L., & Fang, Z. (2022) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3529113.3529114 · Cited by 7
Safe Non-Stochastic Control of Linear Dynamical Systems
Zhou, H., & Tzoumas, V. (2023) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc49753.2023.10383726 · Cited by 7
ONLINE OPTIMAL TASK OFFLOADING WITH ONE-BIT FEEDBACK
Zhao, S., Zhu, Z., Yang, F., & Luo, X. (2018) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2018.8646619 · Cited by 7
Edge resource slicing approaches for latency optimization in AI-edge orchestration
Chandrika, P. K., Mekala, M. S., & Srivastava, G. (2022) · Cluster Computing · doi:10.1007/s10586-022-03817-7 · Cited by 11
“DQL-based models spare energy usage and enhance the lifetime of ED because of pre-classification and estimation of energy and latency rates [59].”
BackgroundQuasi-Stochastic Approximation and Off-Policy Reinforcement Learning
Bernstein, A., Chen, Y., Colombino, M., Dall’Anese, E., Mehta, P. G., & Meyn, S. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029247 · Cited by 6
“Such algorithms have been analyzed extensively using tools similar to the classical SA approach, with similar conclusion on the high variance of the estimates [23].”
MethodElastic job scheduling with unknown utility functions
Fu, X., & Modiano, E. (2021) · Performance Evaluation · doi:10.1016/j.peva.2021.102229 · Cited by 7
Optimal Rate of Convergence for Quasi-Stochastic ApproximationPreprint
Bernstein, A., Chen, Y., Colombino, M., Dall’Anese, E., Mehta, P. G., & Meyn, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07228 · Cited by 8
“Such algorithms have been analyzed extensively using tools similar to the classical SA approach, with similar conclusion on the high variance of the estimates [20].”
MethodEfficiently avoiding saddle points with zero order methods: No gradients requiredPreprint
Flokas, L., Vlatakis-Gkaragkounis, E.-V., & Piliouras, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13021 · Cited by 19
EMI: Energy Management Meets Imputation in Wearable IoT Devices
Hussein, D., Yamin, N., & Bhat, G. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3448379 · Cited by 6
“Indeed, convex optimization for EM in IoT devices has been used in several prior works [35], [36], [37], thus showing its promise in obtaining effective EM decisions.”
MethodHu, C., Zhang, X., & Wu, Q. (2025) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2025.3539529 · Cited by 5
“Similar to [29], [30], we select the optimal input x ( ∗ , k ) ∈ R n as the benchmark sequence at each step k .”
Toward Market-Assisted AI: Cloud Inference for Streamed Data via Model Ensembles From Auctions
Zhang, Y., Jiao, L., Zhu, K., Lin, X., & Zhang, L. (2024) · IEEE Transactions on Networking · doi:10.1109/tnet.2024.3505252 · Cited by 4
“Finally, we design the Ensemble Learning Algorithm (i.e., Algorithm 5) to combine the inference results of the models from Algorithm 4 as a weighted sum to produce the joint result for each data sample, while updating the weights in an exponential manner as data samples arrive [39].”
Dynamic Regret of Quantized Distributed Online Bandit Optimization in Zero-Sum Games
Liao, L., Ho, D. W. C., Yuan, D., Yu, Z., Zhang, B., & Xu, S. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3604774 · Cited by 4
“Additionally, the regret bound in Corollary 1 is comparable to that in [28], which is centralized and also studies the dynamic regret under one-point bandit feedback.”
Dai, Y., & Huang, L. (2024) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3700413 · Cited by 7
“We provide a formal definition of the Bandit Convex Optimization (BCO) problem (Flaxman et al., 2005; Chen and Giannakis, 2018) below.”
InfluentialOn the Convergence of the Inexact Running Krasnosel’skiĭ–Mann Method
Dall’Anese, E., Simonetto, A., & Bernstein, A. (2019) · IEEE Control Systems Letters · doi:10.1109/lcsys.2019.2914085 · Cited by 4
“Errors and time variability of the problem are introduced as follows: • Gradient errors: the gradient of the cost κ(i, s) log(1 + z(i, s)) for each exogenous traffic flow is estimated using a multi-point bandit feedback [15], [26]; the estimation error depends on the number of functional evaluations in constructing th…”
BackgroundMethodBregman-style Online Convex Optimization with EnergyHarvesting Constraints
Asgari, K., & Neely, M. J. (2020) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3428337 · Cited by 4
Towards no regret with no service outages in online resource allocation for edge computing
Chouayakh, A., & Destounis, A. (2022) · International Conference on Communications · doi:10.1109/icc45855.2022.9839038 · Cited by 4
“We compare Algorithm 1 with three algorithms from the literature: the algorithm proposed by Yuan and Lamperski [9] extended for time varying constraints, the algorithm proposed by Yu et al. [18] and the saddle-point method proposed by Chen and Giannakis [19] modified to use full gradient knowledge instead of bandit fe…”
MethodTran‐Dang, H., & Kim, D. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-40978-3_21 · Cited by 3
Safe Pricing Mechanisms for Distributed Resource Allocation With Bandit Feedback
Hutchinson, S., Turan, B., & Alizadeh, M. (2024) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2024.3372143 · Cited by 3
“…algorithms with unknown constraints and feasible iterates where the constraints are either linear [19] or nonlinear [20], as well as online convex optimization with unknown constraints that need to be satisfied in the long term [21] or constraints that need to be satisfied stagewise [22].”
Li, W., Assaad, M., & Zheng, S. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3077516 · Cited by 3
Online Task Offloading with Bandit Learning in Fog-Assisted IoT Systems
Gao, X., Huang, X., & Shao, Z. (2019) · IEEE Vehicular Technology Conference · doi:10.1109/vtcfall.2019.8891376 · Cited by 3
“[8] considered more general scenarios with unknown task latency and developed an effective scheme which incorporates the learning of such uncertainties into offloading decision making.”
BackgroundBayesian Optimization for Task Offloading and Resource Allocation in Mobile Edge Computing
Jia, Y., Lu, Q., & Giannakis, G. B. (2022) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf56349.2022.10051868 · Cited by 3
“Tailored for partial task offloading strategies among multiple edge servers, BCO with both timevarying costs and constraints was studied in [6].”
BackgroundSelf-citationKaya, E. C., Sahin, M. B., & Hashemi, A. (2023) · IEEE Access · doi:10.1109/access.2023.3284891 · Cited by 6
Online bandit convex optimisation with stochastic constraints via two-point feedback
Yu, J., Li, J., & Chen, G. (2023) · International Journal of Systems Science · doi:10.1080/00207721.2023.2209566 · Cited by 3
“Instead, only values of loss functions at the selected action or a number of points close to the action may be informed to the player (Chen & Giannakis, 2018), which is termed as bandit feed-back .”
BackgroundAn Online Algorithm for Data Center Demand Response
Bahrami, S., Chen, Y. C., & Wong, V. W. S. (2019) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2019.8692852 · Cited by 4
“The utility company updates γ1,d(t) as follows [20]:”
BackgroundDistributed Zeroth-Order Stochastic Optimization in Time-varying NetworksPreprint
Li, W., & Assaad, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.12597 · Cited by 3
You, J., Yan, J., Li, Z., & Yang, L. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3674795 · Cited by 5
“Undersuch online bandit setting, the existing approaches typically leverage multi-armed bandit (MAB) [18], [19], and gradient estimation and approximation approaches, e.g., bandit convex optimization (BCO) [20], [21], [22].”
Distributed Constrained Online Optimization with Noisy Communication
Yang, Y.-P., Song, Y., & Yang, S. (2021) · International Conference on Information Science and Technology · doi:10.1109/icist52614.2021.9440593 · Cited by 2
Metadata
- Authors
- Chen, T., & Giannakis, G. B.
- Affiliations
- University of Minnesota (US)
- Venue
- IEEE Internet of Things Journal
- Published
- 2019-02
- Volume
- 6
- Issue
- 1
- Pages
- 1276-1286
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W2739805727
- Cited by
- 139
- References
- 38
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 14 · 2023: 21 · 2022: 16 · 2021: 21 · 2020: 27 · 2019: 23
- FWCI
- 15.48
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1509040, 1508993, 1711471)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Mathematics › Theoretical Computer Science › Optimization and Search Problems
- Keywords
- online convex optimization, bandit feedback, time-varying constraints, dynamic regret, constraint violation, fog computing, computation offloading, Internet of Things, human-in-the-loop systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.