Bandit Convex Optimization for Scalable and Dynamic IoT Management
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
This paper deals with online convex optimization involving both time-varying loss functions, and time-varying constraints. The loss functions are not fully accessible to the learner, and instead only the function values (also known as bandit feedback) are revealed at queried points. The constraints are revealed after making decisions, and can be instantaneously violated, yet they must be satisfied in the long term. This setting fits nicely the emerging online network tasks such as fog computing in the Internet-of-Things, where online decisions must flexibly adapt to the changing user preferences (loss functions), and the temporally unpredictable availability of resources (constraints). Tailored for such human-in-the-loop systems where the loss functions are hard to model, a family of online bandit saddle-point (BanSaP) schemes are developed, which adaptively adjust the online operations based on (possibly multiple) bandit feedback of the loss functions, and the changing environment. Performance here is assessed by: 1) dynamic regret that generalizes the widely used static regret and 2) fit that captures the accumulated amount of constraint violations. Specifically, BanSaP is proved to simultaneously yield sublinear dynamic regret and fit, provided that the best dynamic solutions vary slowly over time. Numerical tests in fog computation offloading tasks corroborate that our proposed BanSaP approach offers competitive performance relative to existing approaches that are based on gradient feedback.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“At this background, large state and action spaces in the whole offloading problem make the conventional learning algorithms [245] [256] [247] infeasible actually.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [247]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2010) · Cited by 8,261
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · cited 3×
“The online computational offloading task of fog computing in IoT [5], [6], [8] takes the form of BCO with long-term constraints (1).”
BackgroundMobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258 · cited 3×
“, N} with moderate processing capability; thus, part of workloads will be collaboratively processed by the local fog servers to meet the stringent latency requirement, and the rest will be offloaded to the remote data center in the cloud layer [6] (see Fig.”
BackgroundEfficient multi-user computation offloading for mobile-edge cloud computing
Chen, X., Jiao, L., Li, W., & Fu, X. (2016) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2487344 · Cited by 2,675 · cited 1×
“Online optimization with time-varying and long-term constraints is well motivated for applications from power control in wireless communication [26], geographical load balancing in cloud networks [14], [27], to computation offloading in fog computing [28], [29].”
BackgroundWeik, M. H. (2000) · Computer Science and Communications Dictionary · doi:10.1007/1-4020-0613-6_12436 · Cited by 2,570
Fog and IoT: An Overview of Research Opportunities
Chiang, M., & Zhang, T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2584538 · Cited by 2,362 · cited 2×
“Consider a mobile network with a sensor layer, a fog layer, and a cloud layer [3], [4].”
BackgroundStochastic Network Optimization with Application to Communication and Queueing Systems
Neely, M. J. (2010) · Synthesis lectures on communication networks · doi:10.2200/s00271ed1v01y201006cnt007 · Cited by 2,218
(2003) · Cited by 1,733
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S. (2012) · now publishers, Inc. eBooks · doi:10.1561/9781601986276 · Cited by 1,404
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S., & Cesa-Bianchi, N. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000024 · Cited by 1,295
Research works printed but not linked: 2
- chen, 2017, Bandit convex optimization for scalable and dynamic IoT management, arXiv preprint 1707 09060
- 2017, 8 Sensors to Help You Create a Smart Home
Cited by
Items 101–138 of 138
Online Convex Optimization for Resource Allocation Scheme in Edge Computing-enabled Networks
Cheng, Y., Li, J., Liang, C., Chai, R., Chen, Q., & Yu, F. R. (2024) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc57260.2024.10571274 · Cited by 2
“AP-cloud : The scheme that application requests can be processed on APs or cloud server [19].”
BackgroundDistributed Event-Triggered Bandit Convex Optimization With Time-Varying Constraints
Zhang, K., Yi, X., Wen, G., Cao, M., Johansson, K. H., Chai, T., & Yang, T. (2025) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2025.3558791 · Cited by 2
“This is reasonable since Chen and Giannakis [15] considered Slater’s condition for constraint functions (i.e., there is a point that strictly satisfies inequality constraints), which is a sufficient condition for strong duality to hold [32]; moreover, the algorithm in [15] uses full-information…”
InfluentialOn Dynamic Regret and Constraint Violations in Constrained Online Convex Optimization
Vaze, R. (2022) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.23919/wiopt56218.2022.9930613 · Cited by 2
“In prior work, starting from [17], where functions f t , g t are assumed to be convex, Lipschitz and smooth, an algorithm has been proposed that achieves √ ) , which was improved in [18], to get x T 3 / 4 ) , and most recently in [19], an algorithm based on the drift plus penalty method has regret…”
BackgroundMethodResultInfluentialKaya, E. C., Sahin, M. B., & Hashemi, A. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096148 · Cited by 2
A Bandit Online Convex Optimization Approach To Distributed Energy Management In Networked Systems
Tsetis, I., Cheng, X., & Maghsudi, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095936 · Cited by 2
“• We evaluate our proposed algorithm using a real-world dataset and compare it with a benchmark named BanSaP [11].”
MethodLazy Queries Can Reduce Variance in Zeroth-Order Optimization
Xiao, Q., Ling, Q. H., & Chen, T. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3316887 · Cited by 3
“Recent advances in this direction include the sign-based ZO method [44], ADMM-based ZO method [18], constrained ZO [45], [46], autoencoder-based ZO [47], ZO for distributed systems [48], [49], [50], [51], diffusion ZO [52] and adaptive sampling ZO by leveraging the inherent sparsity of the gradient…”
MethodSelf-citationYi, X., Li, X., Yang, T., Xie, L., Chai, T., & Johansson, K. H. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147354 · Cited by 2
“The authors of [30] studied online convex optimization with time-varying inequality constraints and bandit feedback for cost functions.”
BackgroundMethodResultInfluentialInformation Aggregation for Constrained Online Control
Li, T., Chen, Y., Sun, B., Wierman, A., & Low, S. H. (2021) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3460085 · Cited by 2
Information Aggregation for Constrained Online Control
Li, T., Chen, Y., Sun, B., Wierman, A., & Low, S. H. (2021) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3543516.3461737 · Cited by 2
Robust Online Learning for Resource Allocation - Beyond Euclidean Projection and Dynamic FitPreprint
Tampubolon, E., & Boche, H. (2019) · arXiv (Cornell University) · doi:10.13140/rg.2.2.34594.58567 · Cited by 2
Yan, X., Li, C., Lu, K., & Xu, H. (2023) · Control Theory and Technology · doi:10.1007/s11768-023-00181-8 · Cited by 1
“For example, in the Internet of Things [15], fog computing can not get the closed expression of delay since its online decision-making needs to adapt to the user preferences and the availability of resources is temporarily unpredictable.”
Reinforcement Learning based Matching for Decentralized Task Offloading in Fog Computing Networks
Tran‐Dang, H., & Kim, D. (2024) · International Conference on Information Networking · doi:10.1109/icoin59985.2024.10572167 · Cited by 1
“The authors in [24] specifically considered the calculation and communication delays of each task as a posterior.”
BackgroundLeveraging Reinforcement Learning for Autonomous Data Pipeline Optimization and Management
Lekkala, C. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr24531190901 · Cited by 1
Guo, H., Zhu, Q., & Liu, X. (2025) · IEEE Transactions on Networking · doi:10.1109/ton.2025.3550732 · Cited by 1
“In [35], bandit convex optimization was introduced for resource management in Internet-of-Things to maximize the users’ utility while satisfying the network resource constraints in the long-term.”
Prediction-Based Distributed Zero-Order Optimization Algorithm Under Sparsification Transmissions
Liu, S., Wang, D., Lian, J., & Wu, F. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3644162 · Cited by 1
“In recent years, IoT technologies have been extensively applied across various fields, such as resource allocation in smart grids [1], machine learning-based sensor estimation [2], and tra ffi c optimization in urban freeways [3].”
Adaptive Budgeted Multi-Armed Bandits for IoT with Dynamic Resource Constraints
Vaishnav, S., Donta, P. K., & Magnússon, S. (2025) · Globecom · doi:10.1109/globecom59602.2025.11432479 · Cited by 1
“Recent studies-such as Chen and Giannakis [9] for energy-critical IoT systems and Liu and Fang [10] for 6G IoT task scheduling—demonstrate the value of incorporating constraints into online learning and decision-making.”
Online convex optimization with switching cost and delayed gradients
Senapati, S., & Vaze, R. (2023) · Performance Evaluation · doi:10.1016/j.peva.2023.102371 · Cited by 1
Distributed No-Regret Learning in Aggregative Games With Residual Bandit Feedback
Liu, W., & Yi, P. (2024) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2024.3395849 · Cited by 1
“However, the required informationmaynotbeavailableduetopracticalconstraints,and explicitlymodelingpayofffunctionsindynamicenvironments ischallenging,suchasservicesatisfactionorreliability[9].”
Online dynamic submodular optimization
Lesage‐Landry, A., & Pallage, J. (2024) · Automatica · doi:10.1016/j.automatica.2024.111758 · Cited by 3
Zeroth-order gradient tracking for decentralized learning with privacy guarantees
Zhao, Z., Xia, L., Jiang, L., Gao, W., & Ge, Q. (2024) · ISA Transactions · doi:10.1016/j.isatra.2024.06.033 · Cited by 2
One-Point Sampling for Distributed Bandit Convex Optimization With Time-Varying Constraints
Zhang, K., Xu, L., Yi, X., Wen, G., Xie, L., Chai, T., & Yang, T. (2025) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2025.3608004 · Cited by 1
“Compared to the algorithms with one-point sampling in [2], [5], [16], and [29], the proposed algorithm does not require knowledge of the total number of iterations.”
InfluentialRobust Online Mirror Saddle-Point Method for Constrained Resource Allocation
Tampubolon, E., & Boche, H. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9052961 · Cited by 1
“Therefore, OL has become a popular method to solve resource allocation and - management problems in engineering fields such as economic dispatch in power systems [2, 3], data center scheduling [4–6], electric vehicle charging [7, 8], video streaming [9], thermal control [10], and fog computing in IoT [11–13].”
BackgroundMethodArtificial Intelligence for Optimizing Edge
Wang, X., Han, Y., Leung, V. C. M., Niyato, D. T., Yan, X., & Chen, X. (2020) · Springer eBooks · doi:10.1007/978-981-15-6186-3_8 · Cited by 1
Information Aggregation for Constrained Online Control
Li, T., Chen, Y., Sun, B., Wierman, A., & Low, S. H. (2021) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/3410220.3461737 · Cited by 1
Online Zeroth-order Optimisation on Hadamard ManifoldsPreprint
Maass, A. I., Manzie, C. G., Nešić, D., Manton, J. H., & Shames, I. (2020) · arXiv (Cornell University) · Cited by 1
Kim, S., Madden, L., & Dall’Anese, E. (2021) · arXiv (Cornell University) · Cited by 1
Online Optimization for Demand Response
Lesage‐Landry, A. (2019) · TSpace (University of Toronto)
Bregman-style Online Convex Optimization with Energy Harvesting Constraints
Asgari, K., & Neely, M. J. (2021) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/3410220.3456270
Wang, J. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/165973
Bandit Learning and Matching-Based Distributed Task Offloading in Fog Networks
Tran‐Dang, H., & Kim, D. (2023) · Springer eBooks · doi:10.1007/978-3-031-33920-2_10
Constrained distributed online convex optimization with bandit feedback for unbalanced digraphs
Tada, K., Hayashi, N., & Takai, S. (2023) · IET Control Theory and Applications · doi:10.1049/cth2.12548
A Learning Approach to Minimum Delay Routing in Stochastic Queueing Networks
Fu, X., & Modiano, E. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10229039
“Such queue length observations enable us to essentially learn D and construct anestimator of the gradients of D following the recently proposed framework of learning in stochastic queueing networks [27], [26].”
BackgroundAn Efficient Bandit Learning based Online Task Offloading in Fog Computing-Enabled Systems
Tran‐Dang, H., & Kim, D. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10311831 · Cited by 1
“The authors in [27] specifically considered the calculation and communication delays of each task as a posteriori.”
Zeroth-Order Methods for Online Distributed Optimization with Strongly Pseudoconvex Cost Functions
Xiaoxi, Y. A. N., Ma, M., & Lu, K. (2024) · 系统科学与信息学报(英文) · doi:10.21078/jssi-2023-0115
Adaptive Bayesian Optimization for Online Management in Mobile Edge Computing
Jia, Y., Lu, Q., & Polyzos, K. D. (2024) · IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM) · doi:10.1109/sam60225.2024.10636403
“These motivate online task offloading and resource allocation in MEC under bandit setting, where only objective function values at queried points are revealed [6]–[8].”
BackgroundHybridization of One- and Two-Point Bandits Convex Optimization in Non-stationary Environments
Zeng, G., & Guo, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1528-5_9
Cheng, Y., Liang, C., Chen, Q., & Yu, F. R. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3673670 · Cited by 3
“In hierarchical computing framework, [20] proposed a bandit saddle point algorithm to optimize task assignment with partial loss function information, and proved that optimal sublinear dynamic regret can be achieved under specific conditions.”
InfluentialDonassolo, B. (2020) · theses.fr (ABES) · doi:10.70675/b9448b30z2deaz4cb9z8ea8z0b5ccd3e535e · Cited by 1
Metadata
- Authors
- Chen, T., & Giannakis, G. B.
- Affiliations
- University of Minnesota (US)
- Venue
- IEEE Internet of Things Journal
- Published
- 2019-02
- Volume
- 6
- Issue
- 1
- Pages
- 1276-1286
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W2739805727
- Cited by
- 139
- References
- 38
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 14 · 2023: 21 · 2022: 16 · 2021: 21 · 2020: 27 · 2019: 23
- FWCI
- 15.48
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1509040, 1508993, 1711471)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Mathematics › Theoretical Computer Science › Optimization and Search Problems
- Keywords
- online convex optimization, bandit feedback, time-varying constraints, dynamic regret, constraint violation, fog computing, computation offloading, Internet of Things, human-in-the-loop systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.