Reliability of SSDs in Enterprise Storage Systems: A Large-Scale Field Study
The DOI matches its Crossref record, with a small difference in title or year; 50 of the 50 references it cites were found in a registry or an index.
Abstract
This article presents the first large-scale field study of NAND-based SSDs in enterprise storage systems (in contrast to drives in distributed data center storage systems). The study is based on a very comprehensive set of field data, covering 1.6 million SSDs of a major storage vendor (NetApp). The drives comprise three different manufacturers, 18 different models, 12 different capacities, and all major flash technologies (SLC, cMLC, eMLC, 3D-TLC). The data allows us to study a large number of factors that were not studied in prior works, including the effect of firmware versions, the reliability of TLC NAND, and the correlations between drives within a RAID system. This article presents our analysis, along with a number of practical implications derived from it.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852 · cited 1×
“In addition to the reliability characteristics of SSDs, prior work has explored the performance characteristics of SSDs under different settings and has shown that their performance can vary significantly from system to system [3, 18, 19, 21, 30, 32, 37, 38, 50, 58, 61, 68].”
BackgroundA case for redundant arrays of inexpensive disks (RAID)
Patterson, D. A., Gibson, G., & Katz, R. H. (1988) · Proceedings of the 1988 ACM SIGMOD international conference on Management of data - SIGMOD '88 · doi:10.1145/50202.50214 · Cited by 1,443
A 3.3 V 32 Mb NAND flash memory with incremental step pulse programming scheme
Suh, K.-D., Suh, B.-H., Lim, Y.-H., Kim, J.-K., Choi, Y.-J., Koh, Y.-N., Lee, S.-S., Kwon, S.-C., Choi, B.-S., Yum, J.-S., et al. (1995) · IEEE Journal of Solid-State Circuits · doi:10.1109/4.475701 · Cited by 515
Write amplification analysis in flash-based solid state drives
Hu, X.-Y., Eleftheriou, E., Haas, R., Iliadis, I., & Pletka, R. (2009) · Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference · doi:10.1145/1534530.1534544 · Cited by 313
An analysis of latent sector errors in disk drives
Bairavasundaram, L. N., Goodson, G. R., Pasupathy, S., & Schindler, J. (2007) · Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems · doi:10.1145/1254882.1254917 · Cited by 296
Chen, F., Lee, R., & Zhang, X. (2011) · 2011 IEEE 17th International Symposium on High Performance Computer Architecture · doi:10.1109/hpca.2011.5749735 · Cited by 283
An analysis of data corruption in the storage stack
Bairavasundaram, L. N., Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., Goodson, G. R., & Schroeder, B. (2008) · ACM Transactions on Storage · doi:10.1145/1416944.1416947 · Cited by 282
Self-citationData retention in MLC NAND flash memory: Characterization, optimization, and recovery
Cai, Y., Luo, Y., Haratsch, E. F., Mai, K., & Mutlu, O. (2015) · 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2015.7056062 · Cited by 274
Flash correct-and-refresh: Retention-aware error management for increased flash memory lifetime
Cai, Y., Yalcin, G., Mutlu, O., Haratsch, E. F., Cristal, A., Unsal, O. S., & Mai, K. (2012) · 2012 IEEE 30th International Conference on Computer Design (ICCD) · doi:10.1109/iccd.2012.6378623 · Cited by 241
Research works printed but not linked: 28
- 2020. NVM Express Specification. https://nvmexpress.org/. [Online ; accessed 23- Oct- 2020 ]. 2020. NVM Express Specification. https://nvmexpress.org/. [Online; accessed 23-Oct-2020].
- Nitin Agrawal , Vijayan Prabhakaran , Ted Wobber , John D. Davis , Mark S. Manasse , and Rina Panigrahy . 2008 . Design tradeoffs for SSD performance . In Proceedings of the USENIX Annual Technical Conference (ATC’08) , Vol. 57 . USENIX Association. Nitin Agrawal, Vijayan Prabhakaran, Ted Wobber, John D. Davis, Mark S. Manasse, and Rina Panigrahy. 2008. Design tradeoffs for SSD performance. In Proceedings of the USENIX Annual Technical Conference (ATC’08), Vol. 57. USENIX Association.
- Alan T. Arnholt and Ben Evans . 2017 . BSDA : Basic Statistics and Data Analysis. https://www.rdocumentation.org/packages/BSDA/versions/1.2.0 R package. Alan T. Arnholt and Ben Evans. 2017. BSDA: Basic Statistics and Data Analysis. https://www.rdocumentation.org/packages/BSDA/versions/1.2.0 R package.
- Matias Bjørling . 2019 . From open-channel SSDs to zoned namespaces . In Proceedings of the Linux Storage and Filesystems Conference (Vault 19) . Matias Bjørling. 2019. From open-channel SSDs to zoned namespaces. In Proceedings of the Linux Storage and Filesystems Conference (Vault 19).
- Yu Cai , Erich F. Haratsch , Onur Mutlu , and Ken Mai . 2012 . Error patterns in MLC NAND flash memory: Measurement, Characterization, and Analysis . In Proceedings of the Conference on Design, Automation and Test in Europe. EDA Consortium, 521–526 . Yu Cai, Erich F. Haratsch, Onur Mutlu, and Ken Mai. 2012. Error patterns in MLC NAND flash memory: Measurement, Characterization, and Analysis. In Proceedings of the Conference on Design, Automation and Test in Europe. EDA Consortium, 521–526.
- Peter Corbett , Bob English , Atul Goel , Tomislav Grcanac , Steven Kleiman , James Leong , and Sunitha Sankar . 2004 . Row-diagonal parity for double disk failure correction . In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST’05) . USENIX Association, 1–14. Peter Corbett, Bob English, Atul Goel, Tomislav Grcanac, Steven Kleiman, James Leong, and Sunitha Sankar. 2004. Row-diagonal parity for double disk failure correction. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST’05). USENIX Association, 1–14.
- John K. Edwards , Daniel Ellard , Craig Everhart , Robert Fair , Eric Hamilton , Andy Kahn , Arkady Kanevsky , James Lentini , Ashish Prakash , Keith A. Smith , and Edward Zayas . 2008 . FlexVol: Flexible, efficient file volume virtualization in WAFL . In Proceedings of the USENIX 2008 Annual Technical Conference (ATC’08) . USENIX Association, 129–142. John K. Edwards, Daniel Ellard, Craig Everhart, Robert Fair, Eric Hamilton, Andy Kahn, Arkady Kanevsky, James Lentini, Ashish Prakash, Keith A. Smith, and Edward Zayas. 2008. FlexVol: Flexible, efficient file volume virtualization in WAFL. In Proceedings of the USENIX 2008 Annual Technical Conference (ATC’08). USENIX Association, 129–142.
- Laura M. Grupp , John D. Davis , and Steven Swanson . 2012 . The bleak future of NAND flash memory . In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12) , Vol. 7 . USENIX Association, 10–2. Laura M. Grupp, John D. Davis, and Steven Swanson. 2012. The bleak future of NAND flash memory. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12), Vol. 7. USENIX Association, 10–2.
- Mingzhe Hao , Gokul Soundararajan , Deepak Kenchammana-Hosekote , Andrew A. Chien , and Haryadi S. Gunawi . 2016. The tail at store: A revelation from millions of hours of disk and SSD deployments . In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16) . USENIX Association, 263–276. Mingzhe Hao, Gokul Soundararajan, Deepak Kenchammana-Hosekote, Andrew A. Chien, and Haryadi S. Gunawi. 2016. The tail at store: A revelation from millions of hours of disk and SSD deployments. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). USENIX Association, 263–276.
- Dave Hitz , James Lau , and Michael A Malcolm . 1994 . File system design for an NFS file server appliance . In USENIX Winter , Vol. 94 . Dave Hitz, James Lau, and Michael A Malcolm. 1994. File system design for an NFS file server appliance. In USENIX Winter, Vol. 94.
- S. Hur , J. Lee , M. Park , J. Choi , K. Park , K. Kim , and K. Kim . 2004. Effective program inhibition beyond 90nm NAND flash memories . Proc. NVSM ( 2004 ), 44–45. S. Hur, J. Lee, M. Park, J. Choi, K. Park, K. Kim, and K. Kim. 2004. Effective program inhibition beyond 90nm NAND flash memories. Proc. NVSM (2004), 44–45.
- 10.5555/2696578.2696591 · doi:10.5555/2696578.2696591
- Ram Kesavan , Jason Hennessey , Richard Jernigan , Peter Macko , Keith A. Smith , Daniel Tennant , and V. R. Bharadwaj . 2019. FlexGroup volumes: A distributed WAFL file system . In Proceedings of the USENIX Annual Technical Conference (ATC’19) . USENIX Association, 135–148. Ram Kesavan, Jason Hennessey, Richard Jernigan, Peter Macko, Keith A. Smith, Daniel Tennant, and V. R. Bharadwaj. 2019. FlexGroup volumes: A distributed WAFL file system. In Proceedings of the USENIX Annual Technical Conference (ATC’19). USENIX Association, 135–148.
- Ram Kesavan , Harendra Kumar , and Sushrut Bhowmik . 2018 . WAFL iron: Repairing live enterprise file systems . In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST’18) . USENIX Association, 33–48. Ram Kesavan, Harendra Kumar, and Sushrut Bhowmik. 2018. WAFL iron: Repairing live enterprise file systems. In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST’18). USENIX Association, 33–48.
- 10.5555/3323298.3323325 · doi:10.5555/3323298.3323325
- 10.5555/3129633.3129651 · doi:10.5555/3129633.3129651
- Farzaneh Mahdisoltani , Ioan Stefanovici , and Bianca Schroeder . 2017 . Proactive error prediction to improve storage system reliability . In USENIX Annual Technical Conference (ATC’17) . USENIX Association, 391–402. Farzaneh Mahdisoltani, Ioan Stefanovici, and Bianca Schroeder. 2017. Proactive error prediction to improve storage system reliability. In USENIX Annual Technical Conference (ATC’17). USENIX Association, 391–402.
- 10.5555/2930583.2930588 · doi:10.5555/2930583.2930588
- Biswaranjan Panda , Deepthi Srinivasan , Huan Ke , Karan Gupta , Vinayak Khot , and Haryadi S Gunawi . 2019 . IASO: A fail-slow detection and mitigation framework for distributed storage services . In Proceedings of the USENIX Annual Technical Conference (ATC’19) . USENIX Association, 47–62. Biswaranjan Panda, Deepthi Srinivasan, Huan Ke, Karan Gupta, Vinayak Khot, and Haryadi S Gunawi. 2019. IASO: A fail-slow detection and mitigation framework for distributed storage services. In Proceedings of the USENIX Annual Technical Conference (ATC’19). USENIX Association, 47–62.
- Stan Park and Kai Shen . 2012 . FIOS: A fair, efficient flash I/O scheduler . In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12) , Vol. 12 . USENIX Association, 13–13. Stan Park and Kai Shen. 2012. FIOS: A fair, efficient flash I/O scheduler. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12), Vol. 12. USENIX Association, 13–13.
- Eduardo Pinheiro , Wolf-Dietrich Weber , and Luiz André Barroso . 2007 . Failure trends in a large disk drive population . In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07) , Vol. 7 . USENIX Association, 17–23. Eduardo Pinheiro, Wolf-Dietrich Weber, and Luiz André Barroso. 2007. Failure trends in a large disk drive population. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07), Vol. 7. USENIX Association, 17–23.
- Abhishek Rajimwale , Vijayan Prabhakaran , and John D. Davis . 2009. Block management in solid-state devices . In Proceedings for the USENIX Annual Technical Conference (ATC’09) . USENIX Association. Abhishek Rajimwale, Vijayan Prabhakaran, and John D. Davis. 2009. Block management in solid-state devices. In Proceedings for the USENIX Annual Technical Conference (ATC’09). USENIX Association.
- Bianca Schroeder and Garth A. Gibson . 2007. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07) , Vol. 7 . USENIX Association, San Jose, CA, 1–16. Bianca Schroeder and Garth A. Gibson. 2007. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07), Vol. 7. USENIX Association, San Jose, CA, 1–16.
- Bianca Schroeder , Raghav Lagisetty , and Arif Merchant . 2016 . Flash reliability in production: The expected and the unexpected . In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16) . USENIX Association, Santa Clara, CA, 67–80. Bianca Schroeder, Raghav Lagisetty, and Arif Merchant. 2016. Flash reliability in production: The expected and the unexpected. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). USENIX Association, Santa Clara, CA, 67–80.
- Thomas Schwarz , Mary Baker , Steven Bassi , Bruce Baumgart , Wayne Flagg , Catherine van Ingen , Kobus Joste , Mark Manasse , and Mehul Shah . 2006 . Disk failure investigations at the internet archive. In Work-in-Progress Session , NASA/IEEE Conference on Mass Storage Systems and Technologies (MSST). Thomas Schwarz, Mary Baker, Steven Bassi, Bruce Baumgart, Wayne Flagg, Catherine van Ingen, Kobus Joste, Mark Manasse, and Mehul Shah. 2006. Disk failure investigations at the internet archive. In Work-in-Progress Session, NASA/IEEE Conference on Mass Storage Systems and Technologies (MSST).
- Dimitris Skourtis , Dimitris Achlioptas , Noah Watkins , Carlos Maltzahn , and Scott Brandt . 2014 . Flash on rails: Consistent flash performance through redundancy . In Proceedings of the USENIX Annual Technical Conference (ATC’14) . USENIX Association, 463–474. Dimitris Skourtis, Dimitris Achlioptas, Noah Watkins, Carlos Maltzahn, and Scott Brandt. 2014. Flash on rails: Consistent flash performance through redundancy. In Proceedings of the USENIX Annual Technical Conference (ATC’14). USENIX Association, 463–474.
- Zoned Storage . [n.d.]. NVMe Zoned Namespaces. https://zonedstorage.io/introduction/zns/. [Online ; accessed 23- Oct- 2020 ]. Zoned Storage. [n.d.]. NVMe Zoned Namespaces. https://zonedstorage.io/introduction/zns/. [Online; accessed 23-Oct-2020].
- Erci Xu , Mai Zheng , Feng Qin , Yikang Xu , and Jiesheng Wu . 2019 . Lessons and actions: What we learned from 10K SSD-related storage system failures . In Proceedings of the USENIX Annual Technical Conference (ATC’19) . USENIX Association, Renton, WA, 961–976. Erci Xu, Mai Zheng, Feng Qin, Yikang Xu, and Jiesheng Wu. 2019. Lessons and actions: What we learned from 10K SSD-related storage system failures. In Proceedings of the USENIX Annual Technical Conference (ATC’19). USENIX Association, Renton, WA, 961–976.
Cited by
(2021) · doi:10.1145/3458336.3465300 · Cited by 55
“This overprovisioning is typically 7–28 of the usable capacity% [10, 32].”
BackgroundRandom Flip Bit Aware Reading for Improving High-Density 3-D NAND Flash Performance
(2024) · doi:10.1109/tcsi.2024.3366902 · Cited by 9
“Flash memory has become the mainstream storage medium for both the consumer market and enterprise users [3], [4], [5].”
BackgroundCloud Uptime Archive: Open-Access Availability Data of Web, Cloud, and Gaming Services
Talluri, S., Niewenhuis, D., Chu, X., Kyselica, J., Cetin, M. B., Balgavy, A., & Iosup, A. (2026) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2026.3673519 · Cited by 4
“In contrast, our community can access much public reliability on hardware failures [1, 2], failures in datacenter (internal) workloads [3, 4], software bug reports [5], and other paradigms such as HPC and networks of desktops [6].”
(2026) · doi:10.1109/tcad.2026.3656765 · Cited by 1
“T HE NAND flash-based solid-state drives (SSDs) have become the core storage solution in data centers and consumer electronics [1], [2] due to their high transfer e ffi ciency, large storage density, and low per-bit cost.”
Metadata
- Authors
- Maneas, S., Mahdaviani, K., Emami, T., & Schroeder, B.
- Affiliations
- University of Toronto (CA)
- NetApp (United States) (US)
- Venue
- ACM Transactions on Storage
- Published
- 2021-01-13
- Volume
- 17
- Issue
- 1
- Pages
- 1-27
- ISSN
- 1553-3077 (ISSN-L)
- 1553-3093
- OpenAlex
- W3118571333
- Cited by
- 13
- References
- 45
- Citations by year
- 2026: 7 · 2025: 1 · 2024: 4 · 2021: 1
- FWCI
- 0.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Storage (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Reliability and Maintenance Optimization
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Keywords
- solid-state drives, NAND flash memory, enterprise storage systems, drive reliability, SLC, RAID, field data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reliability of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. https://app.cochat.ai/papers/10.1145/3423088
- The paper
- Maneas, S., Mahdaviani, K., Emami, T., & Schroeder, B. (2021). Reliability of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. ACM Transactions on Storage. https://doi.org/10.1145/3423088
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.