Low Latency Geo-distributed Data Analytics
The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index.
Abstract
Low latency analytics on geographically distributed datasets (across datacenters, edge clusters) is an upcoming and increasingly important challenge. The dominant approach of aggregating all the data to a single datacenter significantly inflates the timeliness of analytics. At the same time, running queries over geo-distributed inputs using the current intra-DC analytics frameworks also leads to high query response times because these frameworks cannot cope with the relatively low and variable capacity of WAN links. We present Iridium, a system for low latency geo-distributed analytics. Iridium achieves low query response times by optimizing placement of both data and tasks of the queries. The joint data and task placement optimization, however, is intractable. Therefore, Iridium uses an online heuristic to redistribute datasets among the sites prior to queries' arrivals, and places the tasks to reduce network bottlenecks during the query's execution. Finally, it also contains a knob to budget WAN usage. Evaluation across eight worldwide EC2 regions using production queries show that Iridium speeds up queries by 3× -- 19× and lowers WAN usage by 15% -- 64% compared to existing baselines.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
(2010) · Cited by 4,411
Self-citationPrinciples of Distributed Database Systems
Özsu, M. T., & Valduriez, P. (2020) · Springer eBooks · doi:10.1007/978-3-030-26253-2 · Cited by 2,345
B4: experience with a globally-deployed software defined wan
Jain, S., Kumar, A., Mandal, S., Ong, J., Poutievski, L., Singh, A., Venkata, S., Wanderer, J., Zhou, J., Zhu, M., et al. (2013) · Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM · doi:10.1145/2486001.2486019 · Cited by 1,771
Improving MapReduce performance in heterogeneous environments
Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., & Stoica, I. G. (2008) · doi:10.5555/1855741.1855744 · Cited by 1,588
Self-citationDelay scheduling: a simple technique for achieving locality and fairness in cluster scheduling
Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., & Stoica, I. (2010) · Proceedings of the 5th European conference on Computer systems · doi:10.1145/1755913.1755940 · Cited by 1,393
Self-citationA comparison of approaches to large-scale data analysis
Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., & Stonebraker, M. (2009) · Proceedings of the 2009 ACM SIGMOD International Conference on Management of data · doi:10.1145/1559845.1559865 · Cited by 1,071
Achieving high utilization with software-driven WAN
Hong, C.-Y., Kandula, S., Mahajan, R., Zhang, M., Gill, V., Nanduri, M., & Wattenhofer, R. (2013) · Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM · doi:10.1145/2486001.2486012 · Cited by 1,057
Self-citationDiscretized streams: fault-tolerant streaming computation at scale
Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., & Stoica, I. (2013) · Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles · doi:10.1145/2517349.2522737 · Cited by 982
Self-citationHive - a petabyte scale data warehouse using Hadoop
Thusoo, A., Sarma, J. S., Jain, N., Shao, Z., Chakka, P., Zhang, N., Antony, S., Liu, H., & Murthy, R. (2010) · 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010) · doi:10.1109/icde.2010.5447738 · Cited by 929
Cited by
Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., & Zhao, W. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2683200 · Cited by 2,793
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“However, when moving a tremendous amount of data across the wide area network (WAN), both monetary cost and transmission delay can be prohibitively high, and the privacy leakage can also be a major concern [5].”
BackgroundReal-Time Video Analytics: The Killer App for Edge Computing
Ananthanarayanan, G., Bahl, P., Bodik, P., Chintalapudi, K., Philipose, M., Ravindranath, L., & Sinha, S. (2017) · Computer · doi:10.1109/mc.2017.3641638 · Cited by 496
“This is analogical to big data analytics whose original focus was on executing on a single cluster but now is beginning to execute across geo-distributed clusters [2].”
BackgroundSelf-citationComputation Offloading Toward Edge Computing
Lin, L., Liao, X., Jin, H., & Li, P. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2922285 · Cited by 456
VideoEdge: Processing Camera Streams using Hierarchical Clusters
Hung, C.-C., Ananthanarayanan, G., Bodik, P., Golubchik, L., Yu, M., Bahl, P., & Philipose, M. (2018) · 2018 IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00016 · Cited by 287
“Recent work on wide-area analytics – such as Geode [66], Iridium [49], Clarinet [65], Pixida [37] – propose query optimization across geo-distributed clusters.”
BackgroundSelf-citation
Metadata
- Authors
- Pu, Q., Ananthanarayanan, G., Bodik, P., Kandula, S., Akella, A., Bahl, P., & Stoica, I.
- Affiliations
- University of California, Berkeley (US)
- Microsoft (United States) (US)
- Microsoft Research, Redmond, USA
- University of Wisconsin–Madison (US)
- Venue
- Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication
- Publisher
- ACM
- Published
- 2015-08-17
- Pages
- 421-434
- OpenAlex
- W1969299781
- Cited by
- 246
- References
- 50
- Citations by year
- 2025: 3 · 2024: 5 · 2023: 10 · 2022: 11 · 2021: 24 · 2020: 36 · 2019: 49 · 2018: 41 · 2017: 43 · 2016: 20 · 2015: 4
- FWCI
- 31.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Special Interest Group on Data Communication (publishedVersion)
- Funders
- NSF (CNS-1330308, CNS-1345249, CNS-1302041)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Keywords
- iridium, data placement optimization, task placement, online heuristics, edge clusters
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Low Latency Geo-distributed Data Analytics. https://app.cochat.ai/papers/10.1145/2785956.2787505
- The paper
- Pu, Q., Ananthanarayanan, G., Bodik, P., Kandula, S., Akella, A., Bahl, P., & Stoica, I. (2015). Low Latency Geo-distributed Data Analytics. Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. https://doi.org/10.1145/2785956.2787505
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.