CoChat

Multi-laboratory assessment of reproducibility, qualitative and quantitative performance of SWATH-mass spectrometry

Open access

Published version of a preprint

The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index; 19 of the 48 sources its claims cite were checked for support.

Abstract

Quantitative proteomics employing mass spectrometry is an indispensable tool in life science research. Targeted proteomics has emerged as a powerful approach for reproducible quantification but is limited in the number of proteins quantified. SWATH-mass spectrometry consists of data-independent acquisition and a targeted data analysis strategy that aims to maintain the favorable quantitative characteristics (accuracy, sensitivity, and selectivity) of targeted proteomics at large scale. While previous SWATH-mass spectrometry studies have shown high intra-lab reproducibility, this has not been evaluated between labs. In this multi-laboratory evaluation study including 11 sites worldwide, we demonstrate that using SWATH-mass spectrometry data acquisition we can consistently detect and reproducibly quantify >4000 proteins from HEK293 cells. Using synthetic peptide dilution series, we show that the sensitivity, dynamic range and reproducibility established with SWATH-mass spectrometry are uniformly achieved. This study demonstrates that the acquisition of reproducible quantitative proteomics data by multiple labs is achievable, and broadly serves to increase confidence in SWATH-mass spectrometry data acquisition as a reproducible method for large-scale protein quantification.

Overview

CoChat Summary

TL;DR

This multi-laboratory study shows that SWATH-MS data acquisition can reproducibly detect and quantify thousands of proteins across 11 sites, with quantitative performance approaching targeted proteomics. The authors conclude that SWATH-MS is suitable for large-scale, reproducible protein quantification across laboratories.

Key findings

  • Across 229 SWATH-MS files from 11 laboratories, 4,984 proteins were detected at 1% FDR; the median per file was 4,548 proteins, and 4,077 proteins were detected in >80% of all samples.
  • Median repeatability was 91.6% at the protein level and 79.5% at the peptide level across all sites; within-site median repeatability ranged from 90.0–98.2% (protein) and 79.5–95.5% (peptide).
  • For SIS peptides in sample S4, median intra-day CV was 5.5 ± 2.9%, inter-day CV was 8.9 ± 11.1%, and inter-site CV was 47.3 ± 13.9% without normalization, reduced to 21.3 ± 10.3% after normalization.
  • Protein-level CVs for the 4,077 proteins detected in >80% of samples were 8.3 ± 16.2% intra-day, 11.9 ± 17.2% inter-day, and 22.0 ± 17.4% inter-site; median Pearson correlation across all runs was 0.940, with within-site median 0.971 and minimum pairwise 0.868.

Methods

  • Benchmarking sample: 30 stable isotope-labeled standard peptides partitioned into five groups and serially diluted into 1 µg HEK293 cell lysate, creating samples S1–S5 spanning 0.012–10,000 fmol on column; distributed to 11 laboratories worldwide.
  • Acquisition: SWATH-MS on SCIEX TripleTOF 5600/5600+ systems using 64 variable-width Q1 windows; samples S1–S5 acquired with S4 in technical triplicate on days 1, 3, and 5, yielding 229 SWATH-MS files.
  • Analysis: OpenSWATH with a 10,000+ human protein spectral library, PyProphet for 1% FDR control at peptide query and protein levels, aLFQ for protein abundance inference, MultiQuant for SIS peptide XIC analysis, and median normalization based on HEK293 background peptides.

Limitations

  • Stated: the study did not evaluate variance in sample preparation between sites, so no conclusions can be made on that topic; only one instrument type/model was used, limiting tested variables mainly to site of data acquisition.
  • Stated: high peptide loads at the upper end of the dilution series caused some MS signal saturation; a newer instrument platform (TripleTOF 6600) would be expected to reduce this.
  • Stated/evident: no internal standard was available for protein-level dynamic range, so U2OS copy-number data from a different cell line was used as a surrogate; the study is also limited to HEK293 background and SIS peptides, so generalizability to other sample types and platforms is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-017-00249-5

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Study design and implementation. a A set of 30 SIS peptides partitioned into five groups (A–E, six peptides in each) were diluted into a HEK293 cell lysate to span a large dynamic range. Starting at a different upper concentration for each group, they were threefold diluted into the matrix to cover a concentration range from 12 amol to 10 pmol in 1 µg of cell lysate. This created a set of five samples to be run by SWATH-MS on the TripleTOF 5600/5600+ system at each site. Each sample was run once per day on day 1, 3, and 5, with the exception of sample 4 which was run 3× on each day. b After data acquisition, the 229 SWATH-MS files were assembled centrally and processed using two strategies. The SIS peptide concentration curves were assessed using MultiQuant Software, allowing for the determination of linear dynamic range (LDR), and LLOQs for each peptide. In addition, the intra- and inter-day CVs were determined before and after normalization. The HEK293 proteome matrix data was analyzed using the OpenSWATH pipeline and the Combined Human Assay Library consisting of ~10,000 proteins. The false discovery rate was controlled at the peptide query and protein level using PyProphet. Protein abundances were inferred by summing the top five most abundant fragment ions from the three most abundant peak groups using the aLFQ software. We then used protein abundances to cluster, and compute Pearson correlation coefficients, for all samples from all sites
Fig. 1 Study design and implementation. a A set of 30 SIS peptides partitioned into five groups (A–E, six peptides in each) were diluted into a HEK293 cell lysate to span a large dynamic range. Starting at a different upper concentration for each group, they were threefold diluted into the matrix to cover a concentration range from 12 amol to 10 pmol in 1 µg of cell lysate. This created a set of five samples to be run by SWATH-MS on the TripleTOF 5600/5600+ system at each site. Each sample was run once per day on day 1, 3, and 5, with the exception of sample 4 which was run 3× on each day. b After data acquisition, the 229 SWATH-MS files were assembled centrally and processed using two strategies. The SIS peptide concentration curves were assessed using MultiQuant Software, allowing for the determination of linear dynamic range (LDR), and LLOQs for each peptide. In addition, the intra- and inter-day CVs were determined before and after normalization. The HEK293 proteome matrix data was analyzed using the OpenSWATH pipeline and the Combined Human Assay Library consisting of ~10,000 proteins. The false discovery rate was controlled at the peptide query and protein level using PyProphet. Protein abundances were inferred by summing the top five most abundant fragment ions from the three most abundant peak groups using the aLFQ software. We then used protein abundances to cluster, and compute Pearson correlation coefficients, for all samples from all sitesCC BY · PubMed Central
Fig. 2 A consistent set of proteins is detected across sites. a The number of proteins detected in each of the 229 SWATH-MS analyses is shown ordered by site of data collection and then chronologically by time of acquisition. After filtering the data set in a global fashion at 1% FDR at the peptide query and protein levels, a protein was considered detected in a given sample when a peak group for that protein was detected at 1% FDR in the context of that sample (see Supplementary Note 2 for a detailed discussion of FDR). The blue line indicates the cumulate set of proteins detected with each new sample moving from left to right. The maximum of the blue line indicates the set of proteins detected at 1% FDR in the global context. The saturation of the number of proteins detected after a few samples indicates that the set of proteins observed by all sites is highly uniform. b A protein abundance matrix on the log2 scale is shown for 229 SWATH-MS analyses from all sites corresponding to the set of proteins shown in a. White indicates a missing protein abundance value where a given protein was not confidently detected in a given sample. The proteins are ordered from top to bottom first by row completeness and then by protein abundance. c Equivalent to a except that the analysis and FDR control is carried independently out on a site-by-site basis instead of aggregated across all sites before analysis and FDR control
Fig. 2 A consistent set of proteins is detected across sites. a The number of proteins detected in each of the 229 SWATH-MS analyses is shown ordered by site of data collection and then chronologically by time of acquisition. After filtering the data set in a global fashion at 1% FDR at the peptide query and protein levels, a protein was considered detected in a given sample when a peak group for that protein was detected at 1% FDR in the context of that sample (see Supplementary Note 2 for a detailed discussion of FDR). The blue line indicates the cumulate set of proteins detected with each new sample moving from left to right. The maximum of the blue line indicates the set of proteins detected at 1% FDR in the global context. The saturation of the number of proteins detected after a few samples indicates that the set of proteins observed by all sites is highly uniform. b A protein abundance matrix on the log2 scale is shown for 229 SWATH-MS analyses from all sites corresponding to the set of proteins shown in a. White indicates a missing protein abundance value where a given protein was not confidently detected in a given sample. The proteins are ordered from top to bottom first by row completeness and then by protein abundance. c Equivalent to a except that the analysis and FDR control is carried independently out on a site-by-site basis instead of aggregated across all sites before analysis and FDR controlCC BY · PubMed Central
Fig. 3 Reproducibility of SWATH-MS measurements. a The CVs of peak areas for each of the 30 SIS peptides for S4 sample, depicted on the y-axis using logarithmic scaling, were determined at the intra-day level within the site (light blue—without normalization, dark blue with normalization), inter-day level within site (light green—without normalization, dark green with normalization), and inter-site level (i.e., over all S4 samples in the study; light gray—without normalization, dark gray—with normalization). The orange line indicates 20% CV for visual reference. b Similarly, the CV of protein abundances for the 4077 proteins that were detected in >80% all samples were computed at the intra-day level within the site, inter-day with site, and inter-site (i.e., all 229 samples in the study). c The inter-site CVs were binned based on log2 protein abundance to visualize the dependence of CV on protein abundance
Fig. 3 Reproducibility of SWATH-MS measurements. a The CVs of peak areas for each of the 30 SIS peptides for S4 sample, depicted on the y-axis using logarithmic scaling, were determined at the intra-day level within the site (light blue—without normalization, dark blue with normalization), inter-day level within site (light green—without normalization, dark green with normalization), and inter-site level (i.e., over all S4 samples in the study; light gray—without normalization, dark gray—with normalization). The orange line indicates 20% CV for visual reference. b Similarly, the CV of protein abundances for the 4077 proteins that were detected in >80% all samples were computed at the intra-day level within the site, inter-day with site, and inter-site (i.e., all 229 samples in the study). c The inter-site CVs were binned based on log2 protein abundance to visualize the dependence of CV on protein abundanceCC BY · PubMed Central
Fig. 4 Dynamic range and linearity. a The response curves for each of the 30 SIS peptides for Site 1 were determined and plotted together (corresponding plots for all other sites are shown in Supplementary Fig. 13). b From this data, an average response curve for each site was constructed by averaging (mean) the responses of peptides at the same concentration point. This visualization facilitates comparison of both the dynamic range and average response between sites. c The average response curves from b replotted after the normalization has been applied. d The proteins detected in the SWATH-MS analysis of the HEK293 proteome matrix were mapped onto a previous in-depth DDA analysis of the U2OS cell line that employed multi-level fractionation to achieve deep proteome coverage. To demonstrate the dynamic range achieved by the single-shot SWATH-MS analysis we plotted the proteins detected by SWATH-MS binned by the protein copies per cell value (log10 scale) determined from the in-depth U2OS DDA study60. In the range 105−107 copies per cell the proteome coverage is essentially complete and decreases with lower copies per cell bins
Fig. 4 Dynamic range and linearity. a The response curves for each of the 30 SIS peptides for Site 1 were determined and plotted together (corresponding plots for all other sites are shown in Supplementary Fig. 13). b From this data, an average response curve for each site was constructed by averaging (mean) the responses of peptides at the same concentration point. This visualization facilitates comparison of both the dynamic range and average response between sites. c The average response curves from b replotted after the normalization has been applied. d The proteins detected in the SWATH-MS analysis of the HEK293 proteome matrix were mapped onto a previous in-depth DDA analysis of the U2OS cell line that employed multi-level fractionation to achieve deep proteome coverage. To demonstrate the dynamic range achieved by the single-shot SWATH-MS analysis we plotted the proteins detected by SWATH-MS binned by the protein copies per cell value (log10 scale) determined from the in-depth U2OS DDA study60. In the range 105−107 copies per cell the proteome coverage is essentially complete and decreases with lower copies per cell binsCC BY · PubMed Central
Fig. 5 Lower limit of quantification in SWATH-MS and MS1. The percentage of the 30 SIS peptides detected at each concentration in the dilution series from each site of data collection was plotted at the SWATH-MS level a and the MS1 level b. Lower limit of quantification was defined as <20% CV, S/N > 20, 80–120% accuracy using linear fit with 1/x weighting in the response curve. Spectral peak widths for XIC generation were 0.02 m/z for MS1 and 0.05 m/z for SWATH-MS2, and the nominal resolving power was 30,000 and 15,000, respectively. c The average % detection at each concentration for all sites was determined (bold line in a and b) and overlaid to summarize detection differences between SWATH-MS and MS1 data. For the MS1 data, the C12 and C13 XIC data was also summed for comparison. Error bars are ± 1 standard deviation. d The data from a single site (site 1) is also shown for comparison
Fig. 5 Lower limit of quantification in SWATH-MS and MS1. The percentage of the 30 SIS peptides detected at each concentration in the dilution series from each site of data collection was plotted at the SWATH-MS level a and the MS1 level b. Lower limit of quantification was defined as <20% CV, S/N > 20, 80–120% accuracy using linear fit with 1/x weighting in the response curve. Spectral peak widths for XIC generation were 0.02 m/z for MS1 and 0.05 m/z for SWATH-MS2, and the nominal resolving power was 30,000 and 15,000, respectively. c The average % detection at each concentration for all sites was determined (bold line in a and b) and overlaid to summarize detection differences between SWATH-MS and MS1 data. For the MS1 data, the C12 and C13 XIC data was also summed for comparison. Error bars are ± 1 standard deviation. d The data from a single site (site 1) is also shown for comparisonCC BY · PubMed Central
Fig. 6 Clustering and correlation of SWATH-MS quantitative proteomes. a The dendrogram for the 229 samples from all sites resulting from hierarchical clustering based on the log2 protein abundances generated from the SWATH-MS data is shown. The sites are color coded as per the legend. The “D” and “S” notation refers to the day and sample number respectively (Fig 1a). The samples primarily cluster by site of data acquisition whereas the day of data acquisition with one site is generally not clustered. b A correlation matrix showing Pearson coefficients between the 229 samples (all vs. all) is shown. The samples are ordered first by site and then chronologically. The color-scale indicates the magnitude of the Pearson correlation coefficient and the gray arrowheads on the color-scale indicate the median and minimum Pearson correlation across all binary comparisons
Fig. 6 Clustering and correlation of SWATH-MS quantitative proteomes. a The dendrogram for the 229 samples from all sites resulting from hierarchical clustering based on the log2 protein abundances generated from the SWATH-MS data is shown. The sites are color coded as per the legend. The “D” and “S” notation refers to the day and sample number respectively (Fig 1a). The samples primarily cluster by site of data acquisition whereas the day of data acquisition with one site is generally not clustered. b A correlation matrix showing Pearson coefficients between the 229 samples (all vs. all) is shown. The samples are ordered first by site and then chronologically. The color-scale indicates the magnitude of the Pearson correlation coefficient and the gray arrowheads on the color-scale indicate the median and minimum Pearson correlation across all binary comparisonsCC BY · PubMed Central

Verification

Claims the paper makes

10 Valid · 1 Not valid · 16 Not checked · 81 Own results and interpretations

46 more claims

Create a free account to see all 108 claims and to verify the 29 references still open.

See all claims — free account
  • It may state two separate claims as one. 42
  • Probably not a substantial claim. 21

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 82 references

Research works printed but not linked: 1

  1. Ting, Y. S. et al. Peptide-centric proteome analysis: an alternative strategy for the analysis of tandem mass spectrometry data. Mol. Cell. Proteomics 14, 2301–2307 (2015). · DOI · Paper page

Cited by

See all 599

Metadata

Authors
Collins, B. C., Hunter, C. L., Liu, Y., Schilling, B., Rosenberger, G., Bader, S. L., Chan, D. W., Gibson, B. W., Gingras, A.-C., Held, J. M., Hirayama-Kurogi, M., Hou, G., Krisp, C., Larsen, B., Lin, L., Liu, S., Molloy, M. P., Moritz, R. L., Ohtsuki, S., Schlapbach, R., Selevsek, N., Thomas, S. N., Tzeng, S.-C., Zhang, H., & Aebersold, R.
Affiliations
  • ETH Zurich (CH)
  • Buck Institute for Research on Aging (US)
  • University of Zurich (CH)
  • Institute for Systems Biology (US)
  • Johns Hopkins University (US)
  • University of Toronto (CA)
  • Washington University in St. Louis (US)
  • BGI Group (China) (CN)
  • Macquarie University (AU)
  • Lunenfeld-Tanenbaum Research Institute (CA)
  • Kumamoto University (JP)
Corresponding author
Aebersold, Ruedi
Venue
Nature Communications
Published
2017-08-21
Volume
8
Issue
1
Pages
291-291
ISSN
2041-1723 (ISSN-L)
OpenAlex
W2950557897
Cited by
576
References
80
Citations by year
2026: 37 · 2025: 60 · 2024: 77 · 2023: 81 · 2022: 66 · 2021: 71 · 2020: 60 · 2019: 66 · 2018: 45 · 2017: 11
FWCI
25.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Portal (Queen's University Belfast) (publishedVersion, cc-by)
  • Digital Commons@Becker (Washington University School of Medicine) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection (cc-by)
Funders
  • National Science Foundation (1330912, 1S10 OD016281)
  • Genome Canada (OGI-097)
  • Japan Agency for Medical Research and Development (CREST)
  • Ontario Genomics (OGI-097)
  • Buck Institute for Research on Aging
  • Australian Government
  • Government of Canada (FDN 143301, 143301)
  • European Commission (233226, 602156)
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (143301, 166435, 161435)
  • SystemsX.ch
  • National Institutes of Health (1s10od016281-01)
  • Canadian Institutes of Health Research (14330_1, FDN143301)
  • Functional Genomics Center Zurich
  • National Human Genome Research Institute (RC2HG005805, HG005805)
  • National Cancer Institute (U24CA160036)
  • National Institute of General Medical Sciences (R01GM087221, R01GM087221, S10RR027584 and 2P50GM076547, P50GM076547)
  • National Center for Research Resources (S10RR027584)
  • Core Research for Evolutional Science and Technology
  • H2020 European Research Council (670821)
Topics
  • Physical Sciences › Chemistry › Spectroscopy › Advanced Proteomics Techniques and Applications
  • Physical Sciences › Chemistry › Spectroscopy › Mass Spectrometry Techniques and Applications
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Advanced Biosensing Techniques and Applications
Keywords
SWATH mass spectrometry, reproducibility, quantitative proteomics, targeted proteomics, data-independent acquisition, protein quantification, HEK293 cells, sensitivity, dynamic range
MeSH terms
Humans, Laboratories, Mass Spectrometry, Reproducibility of Results, Proteome, Proteomics, HEK293 Cells, Laboratory Proficiency Testing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Multi-laboratory assessment of reproducibility, qualitative and quantitative performance of SWATH-mass spectrometry. https://app.cochat.ai/papers/10.1038/s41467-017-00249-5
The paper
Collins, B. C., Hunter, C. L., Liu, Y., Schilling, B., Rosenberger, G., Bader, S. L., Chan, D. W., Gibson, B. W., Gingras, A.-C., Held, J. M., Hirayama-Kurogi, M., Hou, G., Krisp, C., Larsen, B., Lin, L., Liu, S., Molloy, M. P., Moritz, R. L., Ohtsuki, S., . . . Aebersold, R. (2017). Multi-laboratory assessment of reproducibility, qualitative and quantitative performance of SWATH-mass spectrometry. Nature Communications. https://doi.org/10.1038/s41467-017-00249-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.