The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Reclaiming space from duplicate files in a serverless distributed file system
Abstract
The Farsite distributed file system provides availability by replicating each file onto multiple desktop computers. Since this replication consumes significant storage space, it is important to reclaim used space where possible. Measurement of over 500 desktop file systems shows that nearly half of all consumed space is occupied by duplicate files. We present a mechanism to reclaim space from this incidental duplication to make it available for controlled file replication. Our mechanism includes: (1) convergent encryption, which enables duplicate files to be coalesced into the space of a single file, even if the files are encrypted with different users' keys; and (2) SALAD, a Self-Arranging Lossy Associative Database for aggregating file content and location information in a decentralized, scalable, fault-tolerant manner. Large-scale simulation experiments show that the duplicate-file coalescing system is scalable, highly effective, and fault-tolerant.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Handbook of applied cryptography
(1997) · Choice Reviews Online · doi:10.5860/choice.34-4512 · Cited by 10,357
Chord: A scalable peer-to-peer lookup service for internet applications
Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., & Balakrishnan, H. (2001) · Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications · doi:10.1145/383059.383071 · Cited by 9,623
Space/time trade-offs in hash coding with allowable errors
Bloom, B. H. (1970) · Communications of the ACM · doi:10.1145/362686.362692 · Cited by 7,492
Kocher, P., Jaffe, J., & Jun, B. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-48405-1_25 · Cited by 7,407
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems
Rowstron, A., & Druschel, P. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-45518-3_18 · Cited by 7,276
A scalable content-addressable network
Ratnasamy, S., Francis, P., Handley, M., Karp, R., & Shenker, S. (2001) · Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications · doi:10.1145/383059.383072 · Cited by 6,372
Random oracles are practical: a paradigm for designing efficient protocols
Bellare, M., & Rogaway, P. (1993) · Proceedings of the 1st ACM conference on Computer and communications security - CCS '93 · doi:10.1145/168588.168596 · Cited by 4,712
Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems
Kocher, P. C. (1996) · Lecture Notes in Computer Science · doi:10.1007/3-540-68697-5_9 · Cited by 4,387
Practical techniques for searches on encrypted data
Song, D. X., Wagner, D., & Perrig, A. (2002) · Proceeding 2000 IEEE Symposium on Security and Privacy. S&P 2000 · doi:10.1109/secpri.2000.848445 · Cited by 3,350
Practical Byzantine fault tolerance
Castro, M., & Liskov, B. H. (1999) · Operating Systems Design and Implementation · doi:10.5555/296806.296824 · Cited by 3,205
The Art of Computer Systems Performance Analysis.
Jain, R. (1990) · Int. CMG Conference · Cited by 2,280
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and Routing
Zhao, B. Y., Kubiatowicz, J. D., & Joseph, A. D. (2001) · Cited by 2,146
Freenet: A Distributed Anonymous Information Storage and Retrieval System
Clarke, I., Sandberg, O., Wiley, B., & Hong, T. W. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44702-4_4 · Cited by 2,109
Chord: A scalable peer-to-peer lookup service for internet applications
Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., & Balakrishnan, H. (2001) · ACM SIGCOMM Computer Communication Review · doi:10.1145/964723.383071 · Cited by 1,923
Measurement study of peer-to-peer file sharing systems:
Saroiu, S., Gummadi, P. K., & Gribble, S. D. (2001) · SPIE Proceedings · doi:10.1117/12.449977 · Cited by 1,905
A scalable content-addressable network
Ratnasamy, S., Francis, P., Handley, M., Karp, R., & Shenker, S. (2001) · ACM SIGCOMM Computer Communication Review · doi:10.1145/964723.383072 · Cited by 1,799
Differential fault analysis of secret key cryptosystems
Biham, E., & Adi Shamir, Z. (1997) · Lecture notes in computer science · doi:10.1007/bfb0052259 · Cited by 1,671
Scale and performance in a distributed file system
Howard, J. H., Kazar, M. L., Menees, S. G., Nichols, D. A., Satyanarayanan, M., Sidebotham, R. N., & West, M. J. (1988) · ACM Transactions on Computer Systems · doi:10.1145/35037.35059 · Cited by 1,403
Dynamic Host Configuration Protocol
Droms, R. (1993) · doi:10.17487/rfc1531 · Cited by 1,025
The art of computer systems performance analysis
Chlamtac, I. (1992) · Journal of Molecular Graphics · doi:10.1016/0263-7855(92)80055-i · Cited by 1,020
A low-bandwidth network file system
Muthitacharoen, A., Chen, B., & Mazières, D. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502052 · Cited by 784
Dynamic Host Configuration Protocol
Droms, R. (1997) · doi:10.17487/rfc2131 · Cited by 761
Accessing nearby copies of replicated objects in a distributed environment
Plaxton, C. G., Rajaraman, R., & Richa, A. W. (1997) · Proceedings of the ninth annual ACM symposium on Parallel algorithms and architectures - SPAA '97 · doi:10.1145/258492.258523 · Cited by 760
Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs
Bolosky, W. J., Douceur, J. R., Ely, D., & Theimer, M. (2000) · Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems · doi:10.1145/339331.339345 · Cited by 484
Self-citationSolomon, D. A., & Custer, H. (1992) · Medical Entomology and Zoology · Cited by 292
Publius: a robust, tamper-evident, censorship-resistant web publishing system
Waldman, M., Rubin, A. D., & Cranor, L. F. (2000) · Cited by 281
Side channel cryptanalysis of product ciphers
Kelsey, J., Schneier, B., Wagner, D., & Hall, C. (2000) · Journal of Computer Security · doi:10.3233/jcs-2000-82-304 · Cited by 257
Tapestry: a fault-tolerant wide-area application infrastructure
Zhao, B. Y., Kubiatowicz, J. D., & Joseph, A. D. (2002) · ACM SIGCOMM Computer Communication Review · doi:10.1145/510726.510755 · Cited by 166
Two practical and provably secure block ciphers: BEAR and LION
Anderson, R., & Biham, E. (1996) · Lecture notes in computer science · doi:10.1007/3-540-60865-6_48 · Cited by 122
Towards an archival Intermemory
Goldberg, A. V., & Yianilos, P. N. (2002) · doi:10.1109/adl.1998.670389 · Cited by 108
Optimizing file availability in a secure serverless distributed file system
Douceur, J. R., & Wattenhofer, R. P. (2002) · IEEE Symposium on Reliable Distributed Systems (SRDS) · doi:10.1109/reldis.2001.969727 · Cited by 76
Self-citation
Cited by
Farsite: federated, available, and reliable storage for an incompletely trusted environment
Adya, A., Bolosky, W. J., Castro, M., Cermak, G., Chaiken, R., Douceur, J. R., Howell, J., Lorch, J. R., Theimer, M., & Wattenhofer, R. P. (2002) · ACM SIGOPS Operating Systems Review · doi:10.1145/844128.844130 · Cited by 780
Self-citationDeterministic and Efficiently Searchable Encryption
Bellare, M., Boldyreva, A., & O’Neill, A. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74143-5_30 · Cited by 745
Message-Locked Encryption and Secure Deduplication
Bellare, M., Keelveedhi, S., & Ristenpart, T. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-38348-9_18 · Cited by 632
Secure Deduplication with Efficient and Reliable Convergent Key Management
Li, J., Chen, X., Li, M., Li, J., Lee, P. P. C., & Lou, W. (2014) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2013.284 · Cited by 549
Side Channels in Cloud Services: Deduplication in Cloud Storage
Harnik, D., Pinkas, B., & Shulman-Peleg, A. (2010) · IEEE Security & Privacy Magazine · doi:10.1109/msp.2010.187 · Cited by 522
Metadata
- Authors
- Douceur, J. R., Adya, A., Bolosky, W. J., Simon, P., & Theimer, M.
- Venue
- Proceedings 22nd International Conference on Distributed Computing Systems
- Publisher
- IEEE Comput. Soc
- Published
- 2003-06-25
- Pages
- 617-624
- OpenAlex
- W2157240622
- Cited by
- 645
- References
- 31
- FWCI
- 3.25
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.