Performance of neural network basecalling tools for Oxford Nanopore sequencing
Open accessPublished version of a preprint
Abstract
BACKGROUND: Basecalling, the computational process of translating raw electrical signal to nucleotide sequence, is of critical importance to the sequencing platforms produced by Oxford Nanopore Technologies (ONT). Here, we examine the performance of different basecalling tools, looking at accuracy at the level of bases within individual reads and at majority-rule consensus basecalls in an assembly. We also investigate some additional aspects of basecalling: training using a taxon-specific dataset, using a larger neural network model and improving consensus basecalls in an assembly by additional signal-level analysis with Nanopolish. RESULTS: Training basecallers on taxon-specific data results in a significant boost in consensus accuracy, mostly due to the reduction of errors in methylation motifs. A larger neural network is able to improve both read and consensus accuracy, but at a cost to speed. Improving consensus sequences ('polishing') with Nanopolish somewhat negates the accuracy differences in basecallers, but pre-polish accuracy does have an effect on post-polish accuracy. CONCLUSIONS: Basecalling accuracy has seen significant improvements over the last 2 years. The current version of ONT's Guppy basecaller performs well overall, with good accuracy and fast performance. If higher accuracy is required, users should consider producing a custom model using a larger neural network and/or training data from the same species.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cleaned snRNA-seq object was normalized using SCTransform with mitochondrial percent and RNA UMI counts regressed out.
“The cleaned object was normalized using SCTransform 40 with regressing out mitochondrial percent and RNA UMI counts.”
From Targeting the Immune-Fibrosis Axis in Myocardial Infarction and Heart Failure (2022) · cites this paper as [40]
The Methods > Read sets section describes Oxford Nanopore DNA reads and basecaller benchmarking, not snRNA-seq normalization. Searches for SCTransform, mitochondrial percentage, RNA UMI counts, and single-nucleus RNA normalization found no passage addressing the claimed procedure. No page numbers are available in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measuredThe sequencing accuracy of 84.21% was acceptable compared with previously reported results.
“The accuracy was acceptable compared with the previously reported results (48–50).”
From Electrochemical DNA synthesis and sequencing on a single electrode with scalability for integrated data storage (2021) · cites this paper as [49]
The parameter settings of HVG selection and PCA in these methods are far from consensus and inevitably introduce artificial bias for performance evaluation.
“Furthermore, the parameter settings of HVG selection and PCA in these methods are far to reach a consensus and inevitably introduce artificial bias for performance evaluation 15- 19 .”
From scBERT as a Large-scale Pretrained Deep Language Model for Cell Type Annotation of Single-cell RNA-seq Data (2021) · cites this paper as [16]
HVGs are variable across different batches and different datasets, hindering generalization ability across cohorts.
“However, HVGs are variable across different batches and different datasets, hindering their generalization ability across cohorts 16 .”
From scBERT as a Large-scale Pretrained Deep Language Model for Cell Type Annotation of Single-cell RNA-seq Data (2021) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Minimap2: Pairwise alignment for nucleotide sequences
Li, H. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty191 · Cited by 14,807 · see citing works
Initial sequencing and comparative analysis of the mouse genome
Mouse Genome Sequencing Consortium (2002) · Nature · doi:10.1038/nature01262 · Cited by 7,187 · see citing works
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370 · see citing works
Versatile and open software for comparing large genomes
Kurtz, S., Phillippy, A., Delcher, A. L., Smoot, M., Shumway, M., Antonescu, C., & Salzberg, S. L. (2004) · Genome Biology · doi:10.1186/gb-2004-5-2-r12 · Cited by 5,254 · see citing works
Fast and accurate de novo genome assembly from long uncorrected readsPreprint
Vaser, R., Sović, I., Nagarajan, N., & Šikić, M. (2017) · Genome Research · doi:10.1101/gr.214270.116 · Cited by 3,059 · see citing works
Nanopore sequencing and assembly of a human genome with ultra-long reads
Jain, M., Koren, S., Miga, K. H., Quick, J., Rand, A. C., Sasani, T. A., Tyson, J. R., Beggs, A. D., Dilthey, A. T., Fiddes, I. T., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4060 · Cited by 1,892 · see citing works
A complete bacterial genome assembled de novo using only nanopore sequencing data
Loman, N., Quick, J., & Simpson, J. (2015) · Nature Methods · doi:10.1038/nmeth.3444 · Cited by 1,248 · see citing works
Highly parallel direct RNA sequencing on an array of nanopores
Garalde, D. R., Snell, E. A., Jachimowicz, D., Sipos, B., Lloyd, J. H., Bruce, M., Pantic, N., Admassu, T., James, P. L., Warland, A., et al. (2018) · Nature Methods · doi:10.1038/nmeth.4577 · Cited by 1,094 · see citing works
Piacenza, E., Presentato, A., Ambrosi, E., Speghini, A., Turner, R. J., Vallini, G., & Lampis, S. (2018) · Frontiers in Microbiology · doi:10.3389/fmicb.2018.03178 · Cited by 959 · see citing works
Completing bacterial genome assemblies with multiplex MinION sequencing
Wick, R. R., Judd, L. M., Gorrie, C. L., & Holt, K. E. (2017) · Microbial Genomics · doi:10.1099/mgen.0.000132 · Cited by 956 · see citing works
Self-citationMessenger RNA modifications: Form, distribution, and function
Gilbert, W. V., Bell, T. A., & Schaening-Burgos, C. (2016) · Science · doi:10.1126/science.aad8711 · Cited by 630 · see citing works
SKESA: strategic k-mer extension for scrupulous assemblies
Souvorov, A., Agarwala, R., & Lipman, D. (2018) · Genome Biology · doi:10.1186/s13059-018-1540-z · Cited by 622 · see citing works
Yoshida, C. E., Kruczkiewicz, P., Laing, C., Lingohr, E. J., Gannon, V. P. J., Nash, J. J., & Taboada, E. N. (2016) · PLoS ONE · doi:10.1371/journal.pone.0147101 · Cited by 604 · see citing works
Pightling, A., Pettengill, J., Luo, Y., Baugher, J. D., Rand, H. A., & Strain, E. A. (2018) · Frontiers in Microbiology · doi:10.3389/fmicb.2018.01482 · Cited by 275 · see citing works
DeepNano: Deep recurrent neural networks for base calling in MinION nanopore reads
Boža, V., Brejová, B., & Vinař, T. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178751 · Cited by 246 · see citing works
Chiron: translating nanopore raw signal directly into nucleotide sequence using deep learning
Teng, H., Cao, M. D., Hall, M. B., Duarte, T., Wang, S., & Coin, L. (2018) · GigaScience · doi:10.1093/gigascience/giy037 · Cited by 189 · see citing works
Deepbinner: Demultiplexing barcoded Oxford Nanopore reads with deep convolutional neural networks
Wick, R. R., Judd, L. M., & Holt, K. E. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006583 · Cited by 186 · see citing works
Self-citationGorrie, C. L., Mirčeta, M., Wick, R. R., et al. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/ciy027 · Cited by 141 · see citing works
Self-citationWorkman, R. E., Myrka, A. M., Wong, G. W., Tseng, E., Welch, K. C., & Timp, W. (2018) · GigaScience · doi:10.1093/gigascience/giy009 · Cited by 88 · see citing works
Schjørring, S., Lassen, S. G., Jensen, T., et al. (2017) · Eurosurveillance · doi:10.2807/1560-7917.es.2017.22.50.17-00762 · Cited by 79 · see citing works
Mayston, R., Lloyd-Sherlock, P. G., Gallardo, S., Wang, H., Huang, Y., de, V. M., Ezeah, P., Guerra, M., Sosa, A. L., Liu, Z., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0182360 · Cited by 65 · see citing works
BasecRAWller: Streaming Nanopore Basecalling Directly from Raw SignalPreprint
Stoiber, M. H., & Brown, J. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/133058 · Cited by 37 · see citing works
Rapid Diagnosis of Lower Respiratory Infection using Nanopore-based Clinical Metagenomics
Charalampous, T., Richardson, H., Kay, G. L., Baldan, R., Jeanes, C., Rae, D., Grundy, S., Turner, D. J., Wain, J., Leggett, R. M., et al. (2018) · doi:10.1101/387548 · Cited by 37 · see citing works
Gómez‐Eichelmann, M. C., Levy-Mustri, A., & Ramírez‐Santos, J. (1991) · Journal of Bacteriology · doi:10.1128/jb.173.23.7692-7694.1991 · Cited by 32 · see citing works
Comparison Of Oxford Nanopore Basecalling Tools
Wick, R. R., Judd, L. M., & Holt, K. E. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1043611 · Cited by 24 · see citing works
Self-citationWick, R. (2019) · Figshare · doi:10.26180/5c5a5fa08bbee · Cited by 2 · see citing works
Self-citationWick, R. (2019) · Figshare · doi:10.26180/5c5a5fb6ac10f · Cited by 1 · see citing works
Self-citationWick, R. (2019) · Figshare · doi:10.26180/5c5a5f5ff20ed · Cited by 1 · see citing works
Self-citationWick, R. (2019) · Figshare · doi:10.26180/5c5a7292227de · Cited by 1 · see citing works
Self-citationWick, R. (2019) · Figshare · doi:10.26180/5c5a5fc61e7fa · Cited by 1 · see citing works
Self-citationWick, R. (2019) · Figshare · doi:10.26180/5c5a5fcf72e40 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Opportunities and challenges in long-read sequencing data analysis
Amarasinghe, S. L., Su, S., Dong, X., Zappia, L., Ritchie, M. E., & Gouil, Q. (2020) · Genome Biology · doi:10.1186/s13059-020-1935-5 · Cited by 2,530 · see citing works
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., & Yao, J. (2021) · doi:10.1101/2021.12.05.471261 · Cited by 617 · see citing works
Towards population-scale long-read sequencing
De Coster, W., Weissensteiner, M. H., & Sedlazeck, F. J. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00367-3 · Cited by 323 · see citing works
Predicting RNA-seq coverage from DNA sequence as a unifying model of gene regulation
Linder, J., Srivastava, D., Yuan, H., Agarwal, V., & Kelley, D. R. (2025) · Nature Genetics · doi:10.1038/s41588-024-02053-6 · Cited by 265 · see citing works
Nick, J. A., Dedrick, R. M., Gray, A. L., Vladar, E. K., Smith, B. E., Freeman, K. G., Malcolm, K. C., Epperson, L. E., Hasan, N. A., Hendrix, J., et al. (2022) · Cell · doi:10.1016/j.cell.2022.04.024 · Cited by 251 · see citing works
Curated variation benchmarks for challenging medically relevant autosomal genes
Wagner, J., Olson, N. D., Harris, L., McDaniel, J., Cheng, H., Fungtammasan, A., Hwang, Y.-C., Gupta, R., Wenger, A. M., Rowell, W. J., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01158-1 · Cited by 241 · see citing works
Cellsnp-lite: an efficient tool for genotyping single cells
Huang, X., & Huang, Y. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab358 · Cited by 214 · see citing works
Normalizing and denoising protein expression data from droplet-based single cell profiling
Mulè, M. P., Martins, A. J., & Tsang, J. S. (2022) · Nature Communications · doi:10.1038/s41467-022-29356-8 · Cited by 197 · see citing works
BiT age: A transcriptome‐based aging clock near the theoretical limit of accuracy
Meyer, D. H., & Schumacher, B. (2021) · Aging Cell · doi:10.1111/acel.13320 · Cited by 165 · see citing works
A multi-pronged human microglia response to Alzheimer’s disease Aβ pathology
Mancuso, R., Fattorelli, N., Martinez-Muriana, A., Davis, E., Wolfs, L., Van Den Daele, J., Geric, I., Preman, P., Serneels, L., Poovathingal, S., et al. (2022) · doi:10.1101/2022.07.07.499139 · Cited by 169 · see citing works
DNA language models are powerful predictors of genome-wide variant effects
Benegas, G., Batra, S. S., & Song, Y. S. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311219120 · Cited by 157 · see citing works
Comparison of Transformations for Single-Cell RNA-Seq Data
Ahlmann-Eltze, C., & Huber, W. (2021) · doi:10.1101/2021.06.24.449781 · Cited by 148 · see citing works
Pearman, W. S., Freed, N. E., & Silander, O. K. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3528-4 · Cited by 114 · see citing works
Karakaslar, E. O., Katiyar, N., Hasham, M., Youn, A., Sharma, S., Chung, C., Marches, R., Korstanje, R., Banchereau, J., & Ucar, D. (2023) · Aging Cell · doi:10.1111/acel.13792 · Cited by 98 · see citing works
A panoramic view of cell population dynamics in mammalian aging
Zhang, Z., Schaefer, C., Jiang, W., Lu, Z., Lee, J., Sziraki, A., Abdulraouf, A., Wick, B., Haeussler, M., Li, Z., et al. (2025) · Science · doi:10.1126/science.adn3949 · Cited by 95 · see citing works
Xu, C., Ma, B., Gao, Z., Dong, X., Zhao, C., & Liu, H. (2021) · Science Advances · doi:10.1126/sciadv.abk0100 · Cited by 79 · see citing works
Evaluating the Utilities of Foundation Models in Single-cell Data Analysis
Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H. (2023) · doi:10.1101/2023.09.08.555192 · Cited by 62 · see citing works
Squidiff: predicting cellular development and responses to perturbations using a diffusion model
He, S., Zhu, Y., Tavakol, D. N., Ye, H., Lao, Y.-H., Zhu, Z., Xu, C., Chauhan, S., Garty, G., Tomer, R., et al. (2026) · Nature Methods · doi:10.1038/s41592-025-02877-y · Cited by 38 · see citing works
Liu, B., Polack, M., Coudray, N., Claudio Quiros, A., Sakellaropoulos, T., Le, H., Karimkhan, A., Crobach, A. S. L. P., van Krieken, J. H. J. M., Yuan, K., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57541-y · Cited by 25 · see citing works
Targeting the Immune-Fibrosis Axis in Myocardial Infarction and Heart Failure
Amrute, J. M., Luo, X., Penna, V., Bredemeyer, A., Yamawaki, T., Heo, G. S., Shi, S., Koenig, A., Yang, S., Kadyrov, F., et al. (2022) · bioRxiv · doi:10.1101/2022.10.17.512579 · Cited by 14 · see citing works
Defining Cardiac Recovery at Single Cell Resolution
Amrute, J. M., Lai, L., Ma, P., Koenig, A. L., Kamimoto, K., Bredemeyer, A., Shankar, T. S., Kuppe, C., Kadyrov, F. F., Schulte, L. J., et al. (2022) · doi:10.1101/2022.09.11.507463 · Cited by 6 · see citing works
Robust identification of perturbed cell types in single-cell RNA-seq data
Nicol, P. B., Paulson, D., Qian, G., Liu, X. S., Irizarry, R., & Sahu, A. D. (2023) · doi:10.1101/2023.05.06.539326 · Cited by 33 · see citing works
Zinati, Y., Takiddeen, A., & Emad, A. (2023) · doi:10.1101/2023.07.25.550225 · Cited by 36 · see citing works
- Authors
- Wick, R. R., Judd, L. M., & Holt, K. E.
- Affiliations
- Monash University (AU)
- London School of Hygiene & Tropical Medicine (GB)
- Corresponding author
- Wick, Ryan R.
- Venue
- Genome Biology
- Published
- 2019-06-24
- Volume
- 20
- Issue
- 1
- Pages
- 129-129
- ISSN
- 1465-6906 (ISSN-L)
- 1474-760X
- PMC
- PMC6591954
- OpenAlex
- W2950993016
- Cited by
- 3,221
- References
- 31
- Citations by year
- 2026: 300 · 2025: 577 · 2024: 601 · 2023: 537 · 2022: 524 · 2021: 428 · 2020: 214 · 2019: 30 · 2018: 7 · 2016: 1
- FWCI
- 99.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Genome biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Monash University Research Portal (Monash University) (acceptedVersion, cc-by)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Bill and Melinda Gates Foundation (OPP1175797)
- Sylvia and Charles Viertel Charitable Foundation
- Australian Government Research Training Program Scholarship
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Physical Sciences › Engineering › Biomedical Engineering › Nanopore and Nanochannel Transport Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
- Keywords
- base calling, Oxford Nanopore sequencing, Poecilia reticulata, Nanopolish, model size trade-offs
- MeSH terms
- Klebsiella pneumoniae, Software, Neural Networks, Computer, Sequence Analysis, DNA, Nanopores
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
