A systematic evaluation of data processing and problem formulation of CRISPR off-target site prediction
Open accessPublished version of a preprint
Abstract
CRISPR/Cas9 system is widely used in a broad range of gene-editing applications. While this editing technique is quite accurate in the target region, there may be many unplanned off-target sites (OTSs). Consequently, a plethora of computational methods have been developed to predict off-target cleavage sites given a guide RNA and a reference genome. However, these methods are based on small-scale datasets (only tens to hundreds of OTSs) produced by experimental techniques to detect OTSs with a low signal-to-noise ratio. Recently, CHANGE-seq, a new in vitro experimental technique to detect OTSs, was used to produce a dataset of unprecedented scale and quality (>200 000 OTS over 110 guide RNAs). In addition, the same study included in cellula GUIDE-seq experiments for 58 of the guide RNAs. Here, we fill the gap in previous computational methods by utilizing these data to systematically evaluate data processing and formulation of the CRISPR OTSs prediction problem. Our evaluations show that data transformation as a pre-processing phase is critical prior to model training. Moreover, we demonstrate the improvement gained by adding potential inactive OTSs to the training datasets. Furthermore, our results point to the importance of adding the number of mismatches between guide RNAs and their OTSs as a feature. Finally, we present predictive off-target in cellula models based on both in vitro and in cellula data and compare them to state-of-the-art methods in predicting true OTSs. Our conclusions will be instrumental in any future development of an off-target predictor based on high-throughput datasets.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LNP delivery in vivo can be quickly degraded.
“| Delivery methods improvement | Application in vivo [95–105] | LNP: Can be quickly degraded in vivo | Eliminate off-target effects |”
From Off-target effects in CRISPR/Cas9 gene editing (2023) · cites this paper as [97]
AAV delivery in vivo lasts for years in terminally differentiated cell types.
“| Delivery methods improvement | Application in vivo [95–105] | AAV: Last for years in terminally differentiated cell types | May cause accumulation of off-target mutations |”
From Off-target effects in CRISPR/Cas9 gene editing (2023) · cites this paper as [97]
LNP delivery in vivo eliminates off-target effects.
“| Delivery methods improvement | Application in vivo [95–105] | LNP: Can be quickly degraded in vivo | Eliminate off-target effects |”
From Off-target effects in CRISPR/Cas9 gene editing (2023) · cites this paper as [97]
AAV delivery in vivo may cause accumulation of off-target mutations.
“| Delivery methods improvement | Application in vivo [95–105] | AAV: Last for years in terminally differentiated cell types | May cause accumulation of off-target mutations |”
From Off-target effects in CRISPR/Cas9 gene editing (2023) · cites this paper as [97]
Cites
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · see citing works
Multiplex Genome Engineering Using CRISPR/Cas Systems
Cong, L., Ran, F. A., Cox, D., Lin, S., Barretto, R., Habib, N., Hsu, P. D., Wu, X., Jiang, W., Marraffini, L. A., et al. (2013) · Science · doi:10.1126/science.1231143 · Cited by 15,339 · see citing works
The new frontier of genome engineering with CRISPR-Cas9
Doudna, J. A., & Charpentier, E. (2014) · Science · doi:10.1126/science.1258096 · Cited by 7,062 · see citing works
DNA targeting specificity of RNA-guided Cas9 nucleases
Hsu, P. D., Scott, D. A., Weinstein, J. A., Ran, F. A., Konermann, S., Agarwala, V., Li, Y., Fine, E. J., Wu, X., Shalem, O., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2647 · Cited by 4,741 · see citing works
Optimized sgRNA design to maximize activity and minimize off-target effects of CRISPR-Cas9
Doench, J. G., Fusi, N., Sullender, M., Hegde, M., Vaimberg, E. W., Donovan, K. F., Smith, I., Tothova, Z., Wilen, C., Orchard, R., et al. (2016) · Nature Biotechnology · doi:10.1038/nbt.3437 · Cited by 4,637 · see citing works
Double nicking by RNA-guided CRISPR Cas9 for enhanced genome editing specificity
Ran, F. A., Hsu, P. D., Lin, C.-Y., Gootenberg, J. S., Konermann, S., Trevino, A. E., Scott, D. A., Inoue, A., Matoba, S., Zhang, Y., et al. (2013) · Cell · doi:10.1016/j.cell.2013.08.021 · Cited by 3,299 · see citing works
High-Frequency off-Target Mutagenesis Induced by CRISPR-Cas Nucleases in Human Cells
Fu, Y., Foden, J. A., Khayter, C., Maeder, M. L., Reyon, D., Joung, J. K., & Sander, J. D. (2013) · Nature Biotechnology · doi:10.1038/nbt.2623 · Cited by 3,292 · see citing works
CRISPR/Cas, the Immune System of Bacteria and Archaea
Horvath, P., & Barrangou, R. (2010) · Science · doi:10.1126/science.1179555 · Cited by 2,471 · see citing works
Learning from imbalanced data: open challenges and future directions
Krawczyk, B. (2016) · Progress in Artificial Intelligence · doi:10.1007/s13748-016-0094-0 · Cited by 2,459 · see citing works
Bae, S., Park, J., & Kim, J.-S. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu048 · Cited by 2,339 · see citing works
The CRISPR/Cas bacterial immune system cleaves bacteriophage and plasmid DNA
Garneau, J. E., Dupuis, M.-È., Villion, M., Romero, D. A., Barrangou, R., Boyaval, P., Fremaux, C., Horvath, P., Magadán, A. H., & Moineau, S. (2010) · Nature · doi:10.1038/nature09523 · Cited by 2,336 · see citing works
GUIDE-seq enables genome-wide profiling of off-target cleavage by CRISPR-Cas nucleases
Tsai, S. Q., Zheng, Z., Nguyen, N. T., Liebers, M., Topkar, V. V., Thapar, V., Wyvekens, N., Khayter, C., Iafrate, A. J., Le, L. P., et al. (2015) · Nature Biotechnology · doi:10.1038/nbt.3117 · Cited by 2,268 · see citing works
RNA-guided genetic silencing systems in bacteria and archaea
Wiedenheft, B., Sternberg, S. H., & Doudna, J. A. (2012) · Nature · doi:10.1038/nature10886 · Cited by 1,851 · see citing works
The CRISPR tool kit for genome editing and beyond
Adli, M. (2018) · Nature Communications · doi:10.1038/s41467-018-04252-2 · Cited by 1,642 · see citing works
Yang, H., Wang, H., Shivalila, C. S., Cheng, A. W., Shi, L., & Jaenisch, R. (2013) · Cell · doi:10.1016/j.cell.2013.08.022 · Cited by 1,596 · see citing works
RNA virus interference via CRISPR/Cas13a system in plants
Aman, R., Ali, Z., Butt, H., Mahas, A., Aljedaani, F., Khan, M. Z., Ding, S., & Mahfouz, M. (2018) · Genome Biology · doi:10.1186/s13059-017-1381-1 · Cited by 1,497 · see citing works
Pattanayak, V., Lin, S., Guilinger, J. P., Ma, E., Doudna, J. A., & Liu, D. R. (2013) · Nature Biotechnology · doi:10.1038/nbt.2673 · Cited by 1,487 · see citing works
Field-deployable viral diagnostics using CRISPR-Cas13
Myhrvold, C., Freije, C. A., Gootenberg, J. S., Abudayyeh, O. O., Metsky, H. C., Durbin, A. F., Kellner, M. J., Tan, A. L., Paul, L. M., Parham, L. A., et al. (2018) · Science · doi:10.1126/science.aas8836 · Cited by 1,455 · see citing works
CRISPR-Cas guides the future of genetic engineering
Knott, G. J., & Doudna, J. A. (2018) · Science · doi:10.1126/science.aat5011 · Cited by 1,410 · see citing works
Analysis of off-target effects of CRISPR/Cas-derived RNA-guided endonucleases and nickases
Cho, S. W., Kim, S., Kim, Y., Kweon, J., Kim, H. S., Bae, S., & Kim, J.-S. (2014) · Genome Research · doi:10.1101/gr.162339.113 · Cited by 1,356 · see citing works
CCTop: An intuitive, flexible and reliable CRISPR/Cas9 target prediction tool
Stemmer, M., Thumberger, T., del Sol Keyer, M., Wittbrodt, J., & Mateo, J. L. (2015) · PLoS ONE · doi:10.1371/journal.pone.0124633 · Cited by 1,088 · see citing works
Shi, J., Gao, H., Wang, H., Lafitte, H. R., Archibald, R. L., Yang, M., Hakimi, S. M., Mo, H., & Habben, J. E. (2017) · Plant Biotechnology Journal · doi:10.1111/pbi.12603 · Cited by 1,029 · see citing works
Applications of CRISPR technologies in research and beyond
Barrangou, R., & Doudna, J. A. (2016) · Nature Biotechnology · doi:10.1038/nbt.3659 · Cited by 1,005 · see citing works
Digenome-seq: Genome-wide profiling of CRISPR-cas9 off-target effects in human cells
Kim, D., Bae, S., Park, J., Kim, E., Kim, S., Yu, H. R., Hwang, J., Kim, J.-I., & Kim, J.-S. (2015) · Nature Methods · doi:10.1038/nmeth.3284 · Cited by 992 · see citing works
CRISPR-Cas Systems in Bacteria and Archaea: Versatile Small RNAs for Adaptive Defense and Regulation
Bhaya, D., Davison, M., & Barrangou, R. (2011) · Annual Review of Genetics · doi:10.1146/annurev-genet-110410-132430 · Cited by 914 · see citing works
CIRCLE-Seq: A highly sensitive in vitro screen for genome-wide CRISPR-cas9 nuclease off-targets
Tsai, S. Q., Nguyen, N. T., Malagon-Lopez, J., Topkar, V. V., Aryee, M. J., & Joung, J. K. (2017) · Nature Methods · doi:10.1038/nmeth.4278 · Cited by 814 · see citing works
Genome-wide detection of DNA double-stranded breaks induced by engineered nucleases
Frock, R. L., Hu, J., Meyers, R. M., Ho, Y.-J., Kii, E., & Alt, F. W. (2015) · Nature Biotechnology · doi:10.1038/nbt.3101 · Cited by 673 · see citing works
DeepCRISPR: Optimized CRISPR guide RNA design by deep learning
Chuai, G., Ma, H., Yan, J., Chen, M., Hong, N., Xue, D., Zhou, C., Zhu, C., Chen, K., Duan, B., et al. (2018) · Genome biology · doi:10.1186/s13059-018-1459-4 · Cited by 641 · see citing works
Mapping the genomic landscape of CRISPR-Cas9 cleavage
Cameron, P., Fuller, C. K., Donohoue, P. D., Jones, B. N., Thompson, M. S., Carter, M. M., Gradia, S., Vidal, B., Garner, E., Slorach, E. M., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4284 · Cited by 443 · see citing works
Machine Learning from Imbalanced Data Sets 101
Provost, F. J. (2008) · Faculty Digital Archive (New York University Florence) · Cited by 378 · see citing works
BLISS is a versatile and quantitative method for genome-wide profiling of DNA double-strand breaks
Yan, W. X., Mirzazadeh, R., Garnerone, S., Scott, D., Schneider, M. W., Kallas, T., Custodio, J., Wernersson, E., Li, Y., Gao, L., et al. (2017) · Nature Communications · doi:10.1038/ncomms15058 · Cited by 369 · see citing works
Prediction of off-target activities for the end-to-end design of CRISPR guide RNAs
Listgarten, J., Weinstein, M., Kleinstiver, B. P., Sousa, A. A., Joung, J. K., Crawford, J., Gao, K., Hoang, L., Elibol, M., Doench, J. G., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-017-0178-6 · Cited by 336 · see citing works
Optimized CRISPR guide RNA design for two high-fidelity Cas9 variants by deep learning
Wang, D., Zhang, C., Wang, B., Li, B., Wang, Q., Liu, D., Wang, H., Zhou, Y., Shi, L., Lan, F., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-12281-8 · Cited by 331 · see citing works
In vivo CRISPR editing with no detectable genome-wide off-target mutations
Akçakaya, P., Bobbin, M. L., Guo, J. A., Malagon-Lopez, J., Clement, K., Garcia, S. P., Fellows, M. D., Porritt, M. J., Firth, M. A., Carreras, A., et al. (2018) · Nature · doi:10.1038/s41586-018-0500-9 · Cited by 316 · see citing works
CHANGE-seq reveals genetic and epigenetic effects on CRISPR–Cas9 genome-wide activity
Lazzarotto, C. R., Malinin, N. L., Li, Y., Zhang, R., Yang, Y., Lee, G., Cowley, E., He, Y., Lan, X., Jividen, K., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0555-7 · Cited by 283 · see citing works
Orthologous CRISPR–Cas9 enzymes for combinatorial genetic screens
Najm, F. J., Strand, C., Donovan, K. F., Hegde, M., Sanson, K. R., Vaimberg, E. W., Sullender, M. E., Hartenian, E., Kalani, Z., Fusi, N., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4048 · Cited by 279 · see citing works
Cas9-chromatin binding information enables more accurate CRISPR off-target prediction
Singh, R., Kuscu, C., Quinlan, A., Qi, Y., & Adli, M. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv575 · Cited by 217 · see citing works
Abadi, S., Yan, W. X., Amar, D., & Mayrose, I. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005807 · Cited by 200 · see citing works
Off-target predictions in CRISPR-Cas9 gene editing using deep learning
Lin, J., & Wong, K.-C. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty554 · Cited by 185 · see citing works
Gonatopoulos-Pournatzis, T., Aregger, M., Brown, K. R., Farhangmehr, S., Braunschweig, U., Ward, H. N., Ha, K. C. H., Weiss, A., Billmann, M., Durbic, T., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0437-z · Cited by 142 · see citing works
Tools for experimental and computational analyses of off-target editing by programmable nucleases
Bao, X. R., Pan, Y., Lee, C. M., Davis, T. H., & Bao, G. (2021) · Nature Protocols · doi:10.1038/s41596-020-00431-y · Cited by 106 · see citing works
Lin, J., Zhang, Z., Zhang, S., Chen, J., & Wong, K. (2020) · Advanced Science · doi:10.1002/advs.201903562 · Cited by 101 · see citing works
Deep learning improves the ability of sgRNA off-target propensity prediction
Liu, Q., Cheng, X., Liu, G., Li, B., & Liu, X. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3395-z · Cited by 88 · see citing works
PACVr: plastome assembly coverage visualization in R
Gruenstaeudl, M., & Jenke, N. (2019) · BMC Bioinformatics · doi:10.1186/s12859-020-3475-0 · Cited by 65 · see citing works
Charlier, J., Nadon, R., & Makarenkov, V. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab112 · Cited by 56 · see citing works
crisprSQL: a novel database platform for CRISPR/Cas off-target cleavage assays
Störtz, F., & Minary, P. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa885 · Cited by 37 · see citing works
Benchmarking and integrating genome-wide CRISPR off-target detection and prediction
Yan, J., Xue, D., Chuai, G., Gao, Y., Zhang, G., & Liu, Q. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa930 · Cited by 34 · see citing works
Data imbalance in CRISPR off-target prediction
Gao, Y., Chuai, G., Yu, W., Qu, S., & Liu, Q. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz069 · Cited by 31 · see citing works
Zhang, Y., Long, Y., Yin, R., & Kwoh, C. K. (2020) · IEEE Access · doi:10.1109/access.2020.2989454 · Cited by 31 · see citing works
Predicting CRISPR-Cas9 Off-target with Self-supervised Neural Networks
Chen, D., Shu, W., & Peng, S. (2020) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm49941.2020.9313280 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Off-target effects in CRISPR/Cas9 gene editing
Guo, C., Ma, X., Gao, F., & Guo, Y. (2023) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2023.1143157 · Cited by 625 · see citing works
- Authors
- Yaish, O., Asif, M., & Orenstein, Y.
- Affiliations
- Ben-Gurion University of the Negev (IL)
- Corresponding author
- Orenstein, Yaron
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-05-20
- Volume
- 23
- Issue
- 5
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- OpenAlex
- W4226215757
- Cited by
- 19
- References
- 51
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 7 · 2023: 3
- FWCI
- 1.31
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Briefings in Bioinformatics (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nd)
- PubMed (publishedVersion)
- Funders
- Israel Innovation Authority through the CRISPR-IL Consortium
- Israel Science Foundation (351/22)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › CRISPR and Genetic Engineering
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA regulation and disease
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Keywords
- CRISPR/Cas9, guide RNA, GUIDE-seq, data preprocessing, high-throughput sequencing data, genome editing
- MeSH terms
- Gene Editing, RNA, Guide, CRISPR-Cas Systems, Research Design, CRISPR-Cas Systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
