Statistical practice in high-throughput screening data analysis
Abstract
High-throughput screening is an early critical step in drug discovery. Its aim is to screen a large number of diverse chemical compounds to identify candidate 'hits' rapidly and accurately. Few statistical tools are currently available, however, to detect quality hits with a high degree of confidence. We examine statistical aspects of data preprocessing and hit identification for primary screens. We focus on concerns related to positional effects of wells within plates, choice of hit threshold and the importance of minimizing false-positive and false-negative rates. We argue that replicate measurements are needed to verify assumptions of current methods and to suggest data analysis strategies when assumptions are not met. The integration of replicates with robust statistical methods in primary screens will facilitate the discovery of reliable hits, ultimately improving the sensitivity and specificity of the screening process.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The robust Z-score is sensitive to shifts in median and commonly used as a normalization and strength of perturbation value in image-based phenotypic profiling, but it has not been used for estimating replicate dissimilarity nor compared to other distance metrics.
“The robust Z-score is sensitive to shifts in median, and commonly used as a normalization and strength of perturbation value in the context of image-based phenotypic profiling, but it has not been used for estimating replicate dissimilarity, nor compared to other distance metrics12,30,37.”
From A statistical framework for high-content phenotypic profiling using cellular feature distributions (2022) · cites this paper as [30]
Positional effects due to technical variability manifest as distinct spatial patterns across rows, columns and edges in different plates.
“In contrast, positional effects due to technical variability manifest as distinct spatial patterns across the rows, columns and edges in different plates, a common challenge in multi-well-based assays30–33.”
From A statistical framework for high-content phenotypic profiling using cellular feature distributions (2022) · cites this paper as [30]
Positional effects (edge effects, row/column dependencies, or gradient artifacts) are a persistent factor in multi-well assays and microarray experiments.
“First, positional effects (edge effects, row/column dependencies, or gradient artifacts) are a persistent factor in multi-well assays and microarray experiments30,31,52.”
From A statistical framework for high-content phenotypic profiling using cellular feature distributions (2022) · cites this paper as [30]
Cites
Smyth, G. K. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1027 · Cited by 11,829 · see citing works
Understanding robust and exploratory data analysis
Hoaglin, D. C., Mosteller, F. C., & Tukey, J. W. (1983) · Wiley eBooks · Cited by 1,878 · see citing works
Statistics for experimenters: design, innovation, and discovery
(2006) · Choice Reviews Online · doi:10.5860/choice.43-3429 · Cited by 1,657 · see citing works
Baldi, P., & Long, A. D. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.6.509 · Cited by 1,622 · see citing works
Understanding Robust and Exploratory Data Analysis.
Collett, D., Hoaglin, D. C., Mosteller, F. C., & Tukey, J. W. (1984) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981753 · Cited by 1,154 · see citing works
Lee, M. T., Kuo, F. C., Whitmore, G. A., & Sklar, J. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.18.9834 · Cited by 861 · see citing works
Wright, G. W. J., & Simon, R. M. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg345 · Cited by 662 · see citing works
A survey of sampling from contaminated distributions
Tukey, J. W. (1960) · Medical Entomology and Zoology · Cited by 645 · see citing works
Ratio-based decisions and the quantitative analysis of cDNA microarray images
Chen, Y., Dougherty, E. R., & Bittner, M. L. (1997) · Journal of Biomedical Optics · doi:10.1117/12.281504 · Cited by 551 · see citing works
A Primer on Regression Artifacts
Campbell, D. T., & Kenny, D. A. (1999) · Medical Entomology and Zoology · Cited by 481 · see citing works
Statistics on the table: the history of statistical concepts and methods
(2000) · Choice Reviews Online · doi:10.5860/choice.37-3952 · Cited by 426 · see citing works
Improved Statistical Methods for Hit Selection in High-Throughput Screening
Brideau, C., Gunter, B., Pikounis, B., & Liaw, A. I. (2003) · SLAS DISCOVERY · doi:10.1177/1087057103258285 · Cited by 337 · see citing works
Statistical issues with microarrays: processing and analysis
Nadon, R., & Shoemaker, J. S. (2002) · Trends in Genetics · doi:10.1016/s0168-9525(02)02665-3 · Cited by 297 · see citing works
Self-citationPharmaceutical profiling in drug discovery
Kerns, E. H. (2003) · Drug Discovery Today · doi:10.1016/s1359-6446(03)02649-7 · Cited by 288 · see citing works
Technical Notes: A Simple Technique for Reducing Edge Effect in Cell-Based Assays
Lundholt, B. K., Scudder, K. M., & Pagliaro, L. (2003) · SLAS DISCOVERY · doi:10.1177/1087057103256465 · Cited by 252 · see citing works
Verkman, A. S. (2004) · American Journal of Physiology-Cell Physiology · doi:10.1152/ajpcell.00397.2003 · Cited by 123 · see citing works
Zhang, J.-H., Chung, T. D. Y., & Oldenburg, K. R. (2000) · Journal of Combinatorial Chemistry · doi:10.1021/cc9900706 · Cited by 83 · see citing works
Screening for content—the evolution of high throughput
Dove, A. (2003) · Nature Biotechnology · doi:10.1038/nbt0803-859 · Cited by 68 · see citing works
Gunter, B., Brideau, C., Pikounis, B., et al. (2003) · SLAS DISCOVERY · doi:10.1177/1087057103258284 · Cited by 67 · see citing works
Design and analysis of experiments with high throughput biological assay data
Rocke, D. M. (2004) · Seminars in Cell and Developmental Biology · doi:10.1016/j.semcdb.2004.09.007 · Cited by 59 · see citing works
Landro, J. A., Taylor, I., Stirtan, W. G., Osterman, D. G., Kristie, J. A., Hunnicutt, E. J., Rae, P. M. M., & Sweetnam, P. M. (2000) · Journal of Pharmacological and Toxicological Methods · doi:10.1016/s1056-8719(00)00108-8 · Cited by 47 · see citing works
Design and analysis of experiments with high throughput biological assay data
Rocke, D. M. (2004) · Seminars in Cell and Developmental Biology · doi:10.1016/s1084-9521(04)00093-x · Cited by 44 · see citing works
Invited Perspective: High-Throughput Screening in Academia: The Harvard Experience
Stein, R. L. (2003) · SLAS DISCOVERY · doi:10.1177/1087057103260741 · Cited by 31 · see citing works
Buxser, S. E., & Vroegop, S. M. (2005) · Analytical Biochemistry · doi:10.1016/j.ab.2005.01.034 · Cited by 25 · see citing works
Leveraging process integration in early drug discovery
Fay, N., & Ullmann, D. (2002) · Drug Discovery Today · doi:10.1016/s1359-6446(02)02439-x · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Data-analysis strategies for image-based cell profiling
Caicedo, J. C., Cooper, S., Heigwer, F., Warchal, S., Qiu, P., Molnar, C., Vasilevich, A. S., Barry, J. D., Bansal, H. S., Kraus, O., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4397 · Cited by 739 · see citing works
Statistical methods for analysis of high-throughput RNA interference screens
Birmingham, A., Selfors, L. M., Forster, T., Wrobel, D., Kennedy, C. J., Shanks, E., Santoyo-Lopez, J., Dunican, D. J., Long, A., Kelleher, D., et al. (2009) · Nature Methods · doi:10.1038/nmeth.1351 · Cited by 586 · see citing works
Big data and artificial intelligence modeling for drug discovery
Zhu, H. (2020) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010919-023324 · Cited by 455 · see citing works
Miess, H., Dankworth, B., Gouw, A. M., Rosenfeldt, M., Schmitz, W., Jiang, M., Saunders, B., Howell, M., Downward, J., Felsher, D. W., et al. (2018) · Oncogene · doi:10.1038/s41388-018-0315-z · Cited by 361 · see citing works
Pattern of retinoblastoma pathway inactivation dictates response to CDK4/6 inhibition in GBM
Wiedemeyer, W. R., Dunn, I. F., Quayle, S. N., Zhang, J., Chheda, M. G., Dunn, G. P., Zhuang, L., Rosenbluh, J., Chen, S., Xiao, Y., et al. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1001613107 · Cited by 223 · see citing works
Ciallella, H. L., & Zhu, H. (2019) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.8b00393 · Cited by 227 · see citing works
Zhu, H., Zhang, J., Kim, M. T., Boison, A., Sedykh, A., & Moran, K. (2014) · Chemical Research in Toxicology · doi:10.1021/tx500145h · Cited by 154 · see citing works
An error model for protein quantification
Kreutz, C., Rodriguez, M. M. B., Maiwald, T., Seidl, M., Blum, H. E., Mohr, L., & Timmer, J. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm397 · Cited by 107 · see citing works
Organoids in image-based phenotypic chemical screens
Lukonin, I., Zinner, M., & Liberali, P. (2021) · Experimental & Molecular Medicine · doi:10.1038/s12276-021-00641-8 · Cited by 99 · see citing works
Makarenkov, V., Zentilli, P., Kevorkov, D., Gagarin, A., Malo, N., & Nadon, R. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm145 · Cited by 97 · see citing works
Self-citationHarrill, J. A., Freudenrich, T., Wallace, K., Ball, K., Shafer, T. J., & Mundy, W. R. (2018) · Toxicology and Applied Pharmacology · doi:10.1016/j.taap.2018.04.001 · Cited by 86 · see citing works
Orr, A. L., Ashok, D., Sarantos, M. R., Shi, T., Hughes, R. E., & Brand, M. D. (2013) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2013.08.170 · Cited by 80 · see citing works
Tiziani, S., Kang, Y., Choi, J. S., Roberts, W., & Paternostro, G. (2011) · Nature Communications · doi:10.1038/ncomms1562 · Cited by 60 · see citing works
Novel combinatorial screening identifies neurotrophic factors for selective classes of motor neurons
Schaller, S., Buttigieg, D., Alory, A., Jacquier, A., Barad, M., Merchant, M., Gentien, D., de la Grange, P., & Haase, G. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1615372114 · Cited by 34 · see citing works
A statistical framework for high-content phenotypic profiling using cellular feature distributions
Pearson, Y. E., Kremb, S., Butterfoss, G. L., Xie, X., Fahs, H., & Gunsalus, K. C. (2022) · Communications Biology · doi:10.1038/s42003-022-04343-3 · Cited by 19 · see citing works
“First, positional effects (edge effects, row/column dependencies, or gradient artifacts) are a persistent factor in multi-well assays and microarray experiments 30,31,52 .”
Influential
- Authors
- Malo, N., Hanley, J. A., Cerquozzi, S., Pelletier, J., & Nadon, R.
- Affiliations
- McGill University and Génome Québec Innovation Centre (CA)
- McGill University (CA)
- Corresponding author
- Nadon, Robert
- Venue
- Nature Biotechnology
- Published
- 2006-02
- Volume
- 24
- Issue
- 2
- Pages
- 167-175
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- OpenAlex
- W2097449851
- Cited by
- 721
- References
- 25
- Citations by year
- 2026: 30 · 2025: 25 · 2024: 28 · 2023: 23 · 2022: 21 · 2021: 28 · 2020: 34 · 2019: 46 · 2018: 49 · 2017: 42 · 2016: 38 · 2015: 44 · 2014: 36 · 2013: 42 · 2012: 44
- FWCI
- 33.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Keywords
- high-throughput screening, hit identification, data preprocessing, position effect, false positive rate, false negative rate, replicate measurements, robust statistical methods, drug discovery
- MeSH terms
- Biological Assay, Biometry, Data Interpretation, Statistical, Drug Evaluation, Preclinical, Sensitivity and Specificity, Drug Design, Reproducibility of Results, Guidelines as Topic, Gene Expression Profiling, Microarray Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review