The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
PEGASUS: A Peta-Scale Graph Mining System Implementation and Observations
Abstract
In this paper, we describe PEGASUS, an open source peta graph mining library which performs typical graph mining tasks such as computing the diameter of the graph, computing the radius of each node and finding the connected components. as the size of graphs reaches several giga-, tera- or peta-bytes, the necessity for such a library grows too. To the best of our knowledge, PEGASUS is the first such library, implemented on the top of the HADOOP platform, the open source version of MAPREDUCE. Many graph mining operations (PageRank, spectral clustering, diameter estimation, connected components etc.) are essentially a repeated matrix-vector multiplication. In this paper we describe a very important primitive for PEGASUS, called GIM-V (generalized iterated matrix-vector multiplication). GIM-V is highly optimized, achieving (a) good scale-up on the number of available machines (b) linear running time on the number of edges, and (c) more than 5 times faster performance over the non-optimized version of GIM-V. Our experiments ran on M45, one of the top 50 supercomputers in the world. We report our findings on several real graphs, including one of the largest publicly available Web graphs, thanks to Yahoo!, with ¿ 6,7 billion edges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004
Authoritative sources in a hyperlinked environment
Kleinberg, J. M. (1999) · Journal of the ACM · doi:10.1145/324133.324140 · Cited by 8,959
Newman (2005) · doi:10.1080/00107510500052444 · Cited by 5,749
Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., & Wiener, J. (2000) · Computer Networks · doi:10.1016/s1389-1286(00)00083-9 · Cited by 2,810 · cited 2×
“Graphs are ubiquitous: computer networks, social networks , mobile call networks, the World Wide Web [1], protein regulation networks to name a few.”
BackgroundgSpan: graph-based substructure pattern mining
Yan, X., & Han, J. (2003) · 2002 IEEE International Conference on Data Mining, 2002. Proceedings. · doi:10.1109/icdm.2002.1184038 · Cited by 2,013
Authoritative sources in a hyperlinked environment
Kleinberg, J. M. (1998) · Symposium on Discrete Algorithms · doi:10.5555/314613.315045 · Cited by 1,814
Pig latin: a not-so-foreign language for data processing
Olston, C., Reed, B., Srivastava, U., Kumar, R., & Tomkins, A. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376726 · Cited by 1,730
Grooming, Gossip, and the Evolution of Language
Dunbar, R., & Fisher, M. (1998) · Cited by 796
SCOPE: easy and efficient parallel processing of massive data sets
Chaiken, R., Jenkins, B., Larson, P.-Å., Ramsey, B., Shakib, D., Weaver, S., & Zhou, J. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454166 · Cited by 727
Cited by
Pregel: a system for large-scale graph processing
Malewicz, G., Austern, M. H., Bik, A. J. C., Dehnert, J. C., Horn, I., Leiser, N., & Czajkowski, G. (2010) · Proceedings of the 2010 ACM SIGMOD International Conference on Management of data · doi:10.1145/1807167.1807184 · Cited by 3,515
Distributed GraphLab: a framework for machine learning and data mining in the cloud
Low, Y., Bickson, D., Gonzalez, J., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2212351.2212354 · Cited by 1,653
PowerGraph: distributed graph-parallel computation on natural graphs
Gonzalez, J. E., Low, Y., Gu, H., Bickson, D., & Guestrin, C. (2012) · doi:10.5555/2387880.2387883 · Cited by 1,451
A survey of heterogeneous information network analysis
Shi, C., Li, Y., Zhang, J., Sun, Y., & Yu, P. S. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2598561 · Cited by 1,112
SNAP: A General-Purpose Network Analysis and Graph-Mining Library
Leskovec, J., & Sosič, R. (2017) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2898361 · Cited by 870
GraphChi: Large-Scale Graph Computation on Just a PC
Kyrola, A., Blelloch, G. E., & Guestrin, C. (2018) · Figshare · doi:10.1184/r1/6605999 · Cited by 868
Clustering and community detection in directed networks: A survey
Malliaros, F. D., & Vazirgiannis, M. (2013) · Physics Reports · doi:10.1016/j.physrep.2013.08.002 · Cited by 591
GPS: a graph processing system
Salihoglu, S., & Widom, J. (2013) · Proceedings of the 25th International Conference on Scientific and Statistical Database Management · doi:10.1145/2484838.2484843 · Cited by 510
Data-intensive text processing with MapReduce
Lin, J., & Dyer, C. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Tutorial Abstracts on - NAACL '09 · doi:10.3115/1620950.1620951 · Cited by 445
PIM-enabled instructions: a low-overhead, locality-aware processing-in-memory architecture
Ahn, J., Yoo, S., Mutlu, O., & Choi, K. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2750385 · Cited by 436
Trinity: a distributed graph engine on a memory cloud
Shao, B., Wang, H., & Li, Y. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2467799 · Cited by 432
The Stratosphere platform for big data analytics
Alexandrov, A., Bergmann, R., Ewen, S., Freytag, J.-C., Hueske, F., Heise, A., Kao, O., Leich, M., Leser, U., Markl, V., et al. (2014) · The VLDB Journal · doi:10.1007/s00778-014-0357-y · Cited by 432
One trillion edges: graph processing at Facebook-scale
Ching, A., Edunov, S., Kabiljo, M., Logothetis, D., & Muthukrishnan, S. (2015) · Proceedings of the VLDB Endowment · doi:10.14778/2824032.2824077 · Cited by 404
FENNEL: streaming graph partitioning for massive scale graphs
Tsourakakis, C., Gkantsidis, C., Radunovic, B., & Vojnovic, M. (2014) · Proceedings of the 7th ACM international conference on Web search and data mining · doi:10.1145/2556195.2556213 · Cited by 341
Self-citationSystemML: Declarative machine learning on MapReduce
Ghoting, A., Krishnamurthy, R., Pednault, E., Reinwald, B., Sindhwani, V., Tatikonda, S., Tian, Y., & Vaithyanathan, S. (2011) · 2011 IEEE 27th International Conference on Data Engineering · doi:10.1109/icde.2011.5767930 · Cited by 295
McCune, R. R., Weninger, T., & Madey, G. (2015) · ACM Computing Surveys · doi:10.1145/2818185 · Cited by 294
GraphMat: high performance graph analytics made productive
Sundaram, N., Satish, N., Patwary, M. M. A., Dulloor, S. R., Anderson, M. J., Vadlamudi, S. G., Das, D., & Dubey, P. (2015) · Proceedings of the VLDB Endowment · doi:10.14778/2809974.2809983 · Cited by 282
Graphicionado: A high-performance and energy-efficient accelerator for graph analytics
Ham, T. J., Wu, L., Sundaram, N., Satish, N., & Martonosi, M. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783759 · Cited by 277
Graph Summarization Methods and Applications: A Survey
Liu, Y., Safavi, T., Dighe, A., & Koutra, D. (2019) · ACM Computing Surveys · doi:10.1145/3186727 · Cited by 274
TurboGraph: a fast parallel graph engine handling billion-scale graphs in a single PC
Han, W.-S., Lee, S., Park, K., Lee, J.-H., Kim, M.-S., Kim, J., & Yu, H. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487581 · Cited by 259
Metadata
- Authors
- Kang, U., Tsourakakis, C. E., & Faloutsos, C.
- Venue
- 2009 Ninth IEEE International Conference on Data Mining
- Publisher
- IEEE
- Published
- 2009-12
- Pages
- 229-238
- ISSN
- 1550-4786 (ISSN-L)
- OpenAlex
- W2167927436
- Cited by
- 628
- References
- 32
- FWCI
- 32.71
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.