CoChat

Reliability of SSDs in Enterprise Storage Systems: A Large-Scale Field Study

Maneas, Stathis, Mahdaviani, Kaveh, Emami, Tim, Schroeder, Bianca · 2021 · ACM Transactions on Storage

The DOI matches its Crossref record, with a small difference in title or year; 50 of the 50 references it cites were found in a registry or an index.

Abstract

This article presents the first large-scale field study of NAND-based SSDs in enterprise storage systems (in contrast to drives in distributed data center storage systems). The study is based on a very comprehensive set of field data, covering 1.6 million SSDs of a major storage vendor (NetApp). The drives comprise three different manufacturers, 18 different models, 12 different capacities, and all major flash technologies (SLC, cMLC, eMLC, 3D-TLC). The data allows us to study a large number of factors that were not studied in prior works, including the effect of firmware versions, the reliability of TLC NAND, and the correlations between drives within a RAID system. This article presents our analysis, along with a number of practical implications derived from it.

Overview

Integrity signals

Registry

Crossref · 10.1145/3423088

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 50 references

Research works printed but not linked: 28

  1. 2020. NVM Express Specification. https://nvmexpress.org/. [Online ; accessed 23- Oct- 2020 ]. 2020. NVM Express Specification. https://nvmexpress.org/. [Online; accessed 23-Oct-2020].
  2. Nitin Agrawal , Vijayan Prabhakaran , Ted Wobber , John D. Davis , Mark S. Manasse , and Rina Panigrahy . 2008 . Design tradeoffs for SSD performance . In Proceedings of the USENIX Annual Technical Conference (ATC’08) , Vol. 57 . USENIX Association. Nitin Agrawal, Vijayan Prabhakaran, Ted Wobber, John D. Davis, Mark S. Manasse, and Rina Panigrahy. 2008. Design tradeoffs for SSD performance. In Proceedings of the USENIX Annual Technical Conference (ATC’08), Vol. 57. USENIX Association.
  3. Alan T. Arnholt and Ben Evans . 2017 . BSDA : Basic Statistics and Data Analysis. https://www.rdocumentation.org/packages/BSDA/versions/1.2.0 R package. Alan T. Arnholt and Ben Evans. 2017. BSDA: Basic Statistics and Data Analysis. https://www.rdocumentation.org/packages/BSDA/versions/1.2.0 R package.
  4. Matias Bjørling . 2019 . From open-channel SSDs to zoned namespaces . In Proceedings of the Linux Storage and Filesystems Conference (Vault 19) . Matias Bjørling. 2019. From open-channel SSDs to zoned namespaces. In Proceedings of the Linux Storage and Filesystems Conference (Vault 19).
  5. Yu Cai , Erich F. Haratsch , Onur Mutlu , and Ken Mai . 2012 . Error patterns in MLC NAND flash memory: Measurement, Characterization, and Analysis . In Proceedings of the Conference on Design, Automation and Test in Europe. EDA Consortium, 521–526 . Yu Cai, Erich F. Haratsch, Onur Mutlu, and Ken Mai. 2012. Error patterns in MLC NAND flash memory: Measurement, Characterization, and Analysis. In Proceedings of the Conference on Design, Automation and Test in Europe. EDA Consortium, 521–526.
  6. Peter Corbett , Bob English , Atul Goel , Tomislav Grcanac , Steven Kleiman , James Leong , and Sunitha Sankar . 2004 . Row-diagonal parity for double disk failure correction . In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST’05) . USENIX Association, 1–14. Peter Corbett, Bob English, Atul Goel, Tomislav Grcanac, Steven Kleiman, James Leong, and Sunitha Sankar. 2004. Row-diagonal parity for double disk failure correction. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST’05). USENIX Association, 1–14.
  7. John K. Edwards , Daniel Ellard , Craig Everhart , Robert Fair , Eric Hamilton , Andy Kahn , Arkady Kanevsky , James Lentini , Ashish Prakash , Keith A. Smith , and Edward Zayas . 2008 . FlexVol: Flexible, efficient file volume virtualization in WAFL . In Proceedings of the USENIX 2008 Annual Technical Conference (ATC’08) . USENIX Association, 129–142. John K. Edwards, Daniel Ellard, Craig Everhart, Robert Fair, Eric Hamilton, Andy Kahn, Arkady Kanevsky, James Lentini, Ashish Prakash, Keith A. Smith, and Edward Zayas. 2008. FlexVol: Flexible, efficient file volume virtualization in WAFL. In Proceedings of the USENIX 2008 Annual Technical Conference (ATC’08). USENIX Association, 129–142.
  8. Laura M. Grupp , John D. Davis , and Steven Swanson . 2012 . The bleak future of NAND flash memory . In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12) , Vol. 7 . USENIX Association, 10–2. Laura M. Grupp, John D. Davis, and Steven Swanson. 2012. The bleak future of NAND flash memory. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12), Vol. 7. USENIX Association, 10–2.
  9. Mingzhe Hao , Gokul Soundararajan , Deepak Kenchammana-Hosekote , Andrew A. Chien , and Haryadi S. Gunawi . 2016. The tail at store: A revelation from millions of hours of disk and SSD deployments . In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16) . USENIX Association, 263–276. Mingzhe Hao, Gokul Soundararajan, Deepak Kenchammana-Hosekote, Andrew A. Chien, and Haryadi S. Gunawi. 2016. The tail at store: A revelation from millions of hours of disk and SSD deployments. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). USENIX Association, 263–276.
  10. Dave Hitz , James Lau , and Michael A Malcolm . 1994 . File system design for an NFS file server appliance . In USENIX Winter , Vol. 94 . Dave Hitz, James Lau, and Michael A Malcolm. 1994. File system design for an NFS file server appliance. In USENIX Winter, Vol. 94.
  11. S. Hur , J. Lee , M. Park , J. Choi , K. Park , K. Kim , and K. Kim . 2004. Effective program inhibition beyond 90nm NAND flash memories . Proc. NVSM ( 2004 ), 44–45. S. Hur, J. Lee, M. Park, J. Choi, K. Park, K. Kim, and K. Kim. 2004. Effective program inhibition beyond 90nm NAND flash memories. Proc. NVSM (2004), 44–45.
  12. 10.5555/2696578.2696591 · doi:10.5555/2696578.2696591
  13. Ram Kesavan , Jason Hennessey , Richard Jernigan , Peter Macko , Keith A. Smith , Daniel Tennant , and V. R. Bharadwaj . 2019. FlexGroup volumes: A distributed WAFL file system . In Proceedings of the USENIX Annual Technical Conference (ATC’19) . USENIX Association, 135–148. Ram Kesavan, Jason Hennessey, Richard Jernigan, Peter Macko, Keith A. Smith, Daniel Tennant, and V. R. Bharadwaj. 2019. FlexGroup volumes: A distributed WAFL file system. In Proceedings of the USENIX Annual Technical Conference (ATC’19). USENIX Association, 135–148.
  14. Ram Kesavan , Harendra Kumar , and Sushrut Bhowmik . 2018 . WAFL iron: Repairing live enterprise file systems . In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST’18) . USENIX Association, 33–48. Ram Kesavan, Harendra Kumar, and Sushrut Bhowmik. 2018. WAFL iron: Repairing live enterprise file systems. In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST’18). USENIX Association, 33–48.
  15. 10.5555/3323298.3323325 · doi:10.5555/3323298.3323325
  16. 10.5555/3129633.3129651 · doi:10.5555/3129633.3129651
  17. Farzaneh Mahdisoltani , Ioan Stefanovici , and Bianca Schroeder . 2017 . Proactive error prediction to improve storage system reliability . In USENIX Annual Technical Conference (ATC’17) . USENIX Association, 391–402. Farzaneh Mahdisoltani, Ioan Stefanovici, and Bianca Schroeder. 2017. Proactive error prediction to improve storage system reliability. In USENIX Annual Technical Conference (ATC’17). USENIX Association, 391–402.
  18. 10.5555/2930583.2930588 · doi:10.5555/2930583.2930588
  19. Biswaranjan Panda , Deepthi Srinivasan , Huan Ke , Karan Gupta , Vinayak Khot , and Haryadi S Gunawi . 2019 . IASO: A fail-slow detection and mitigation framework for distributed storage services . In Proceedings of the USENIX Annual Technical Conference (ATC’19) . USENIX Association, 47–62. Biswaranjan Panda, Deepthi Srinivasan, Huan Ke, Karan Gupta, Vinayak Khot, and Haryadi S Gunawi. 2019. IASO: A fail-slow detection and mitigation framework for distributed storage services. In Proceedings of the USENIX Annual Technical Conference (ATC’19). USENIX Association, 47–62.
  20. Stan Park and Kai Shen . 2012 . FIOS: A fair, efficient flash I/O scheduler . In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12) , Vol. 12 . USENIX Association, 13–13. Stan Park and Kai Shen. 2012. FIOS: A fair, efficient flash I/O scheduler. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12), Vol. 12. USENIX Association, 13–13.
  21. Eduardo Pinheiro , Wolf-Dietrich Weber , and Luiz André Barroso . 2007 . Failure trends in a large disk drive population . In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07) , Vol. 7 . USENIX Association, 17–23. Eduardo Pinheiro, Wolf-Dietrich Weber, and Luiz André Barroso. 2007. Failure trends in a large disk drive population. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07), Vol. 7. USENIX Association, 17–23.
  22. Abhishek Rajimwale , Vijayan Prabhakaran , and John D. Davis . 2009. Block management in solid-state devices . In Proceedings for the USENIX Annual Technical Conference (ATC’09) . USENIX Association. Abhishek Rajimwale, Vijayan Prabhakaran, and John D. Davis. 2009. Block management in solid-state devices. In Proceedings for the USENIX Annual Technical Conference (ATC’09). USENIX Association.
  23. Bianca Schroeder and Garth A. Gibson . 2007. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07) , Vol. 7 . USENIX Association, San Jose, CA, 1–16. Bianca Schroeder and Garth A. Gibson. 2007. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07), Vol. 7. USENIX Association, San Jose, CA, 1–16.
  24. Bianca Schroeder , Raghav Lagisetty , and Arif Merchant . 2016 . Flash reliability in production: The expected and the unexpected . In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16) . USENIX Association, Santa Clara, CA, 67–80. Bianca Schroeder, Raghav Lagisetty, and Arif Merchant. 2016. Flash reliability in production: The expected and the unexpected. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). USENIX Association, Santa Clara, CA, 67–80.
  25. Thomas Schwarz , Mary Baker , Steven Bassi , Bruce Baumgart , Wayne Flagg , Catherine van Ingen , Kobus Joste , Mark Manasse , and Mehul Shah . 2006 . Disk failure investigations at the internet archive. In Work-in-Progress Session , NASA/IEEE Conference on Mass Storage Systems and Technologies (MSST). Thomas Schwarz, Mary Baker, Steven Bassi, Bruce Baumgart, Wayne Flagg, Catherine van Ingen, Kobus Joste, Mark Manasse, and Mehul Shah. 2006. Disk failure investigations at the internet archive. In Work-in-Progress Session, NASA/IEEE Conference on Mass Storage Systems and Technologies (MSST).
  26. Dimitris Skourtis , Dimitris Achlioptas , Noah Watkins , Carlos Maltzahn , and Scott Brandt . 2014 . Flash on rails: Consistent flash performance through redundancy . In Proceedings of the USENIX Annual Technical Conference (ATC’14) . USENIX Association, 463–474. Dimitris Skourtis, Dimitris Achlioptas, Noah Watkins, Carlos Maltzahn, and Scott Brandt. 2014. Flash on rails: Consistent flash performance through redundancy. In Proceedings of the USENIX Annual Technical Conference (ATC’14). USENIX Association, 463–474.
  27. Zoned Storage . [n.d.]. NVMe Zoned Namespaces. https://zonedstorage.io/introduction/zns/. [Online ; accessed 23- Oct- 2020 ]. Zoned Storage. [n.d.]. NVMe Zoned Namespaces. https://zonedstorage.io/introduction/zns/. [Online; accessed 23-Oct-2020].
  28. Erci Xu , Mai Zheng , Feng Qin , Yikang Xu , and Jiesheng Wu . 2019 . Lessons and actions: What we learned from 10K SSD-related storage system failures . In Proceedings of the USENIX Annual Technical Conference (ATC’19) . USENIX Association, Renton, WA, 961–976. Erci Xu, Mai Zheng, Feng Qin, Yikang Xu, and Jiesheng Wu. 2019. Lessons and actions: What we learned from 10K SSD-related storage system failures. In Proceedings of the USENIX Annual Technical Conference (ATC’19). USENIX Association, Renton, WA, 961–976.

Cited by

See all 13

Metadata

Authors
Maneas, S., Mahdaviani, K., Emami, T., & Schroeder, B.
Affiliations
  • University of Toronto (CA)
  • NetApp (United States) (US)
Venue
ACM Transactions on Storage
Published
2021-01-13
Volume
17
Issue
1
Pages
1-27
ISSN
  • 1553-3077 (ISSN-L)
  • 1553-3093
OpenAlex
W3118571333
Cited by
13
References
45
Citations by year
2026: 7 · 2025: 1 · 2024: 4 · 2021: 1
FWCI
0.56
Type
journal article
Language
en
Open access
closed
Locations
ACM Transactions on Storage (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Reliability and Maintenance Optimization
  • Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
Keywords
solid-state drives, NAND flash memory, enterprise storage systems, drive reliability, SLC, RAID, field data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reliability of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. https://app.cochat.ai/papers/10.1145/3423088
The paper
Maneas, S., Mahdaviani, K., Emami, T., & Schroeder, B. (2021). Reliability of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. ACM Transactions on Storage. https://doi.org/10.1145/3423088

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.