Predicting protein structures with a multiplayer online game
Open accessAbstract
People exert large amounts of problem-solving effort playing computer games. Simple image- and text-recognition tasks have been successfully 'crowd-sourced' through games, but it is not clear if more complex scientific problems can be solved with human-directed computing. Protein structure prediction is one such problem: locating the biologically relevant native conformation of a protein is a formidable computational challenge given the very large size of the search space. Here we describe Foldit, a multiplayer online game that engages non-scientists in solving hard prediction problems. Foldit players interact with protein structures using direct manipulation tools and user-friendly versions of algorithms from the Rosetta structure prediction methodology, while they compete and collaborate to optimize the computed energy. We show that top-ranked Foldit players excel at solving challenging structure refinement problems in which substantial backbone rearrangements are necessary to achieve the burial of hydrophobic residues. Players working collaboratively develop a rich assortment of new strategies and algorithms; unlike computational approaches, they explore not only the conformational space but also the space of possible search strategies. The integration of human visual problem-solving and strategy development capabilities with traditional computational algorithms through interactive multiplayer games is a powerful new approach to solving computationally-limited scientific problems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Foldit is a popular video game that crowdsources protein structure prediction.
“Foldit is a popular video game that crowdsources protein structure prediction5, challenging players to discover low-energy protein models by exploring protein conformational space.”
From Determining crystal structures through crowdsourcing and coursework (2016) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 9
- von Ahn L Dabbish L Labeling images with a computer game CHI ’04: Proceedings of the 2004 Conference on Human Factors in Computing Systems 319 326 ACM Press 2004 URL http://dx.doi.org/10.1145/985692.985733 · DOI
- von Ahn L Liu R Blum M Peekaboom: a game for locating objects in images CHI ’06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 55 64 ACM New York, NY, USA 2006 URL http://dx.doi.org/10.1145/1124772.1124782 · DOI
- Rohl C Strauss C Misura K Baker D Protein structure prediction using Rosetta Methods in Enzymology 383 66 93 Department of Biochemistry and Howard Hughes Medical Institute, University of Washington, Seattle Washington 98195, USA 2004 URL http://dx.doi.org/10.1016/S0076-6879(04)83004-0 15063647 · DOI
- Anfinsen CB Principles that govern the folding of protein chains Science 181 223 230 1973 URL http://view.ncbi.nlm.nih.gov/pubmed/73220860 4124164
- Das R Baker D Macromolecular modeling with Rosetta Annual Review of Biochemistry 77 363 382 2008 URL http://dx.doi.org/10.1146/annurev.biochem.77.062906.171838 18410248 · DOI
- Qian B High-resolution structure prediction and the crystallographic phase problem Nature 450 259 264 2007 URL http://dx.doi.org/10.1038/nature06249 17934447 PMC2504711 · DOI
- Bradley P Misura KMS Baker D Toward high-resolution de novo structure prediction for small proteins Science 309 1868 1871 2005 URL http://dx.doi.org/10.1126/science.1113801 16166519 · DOI
- Yee N Motivations of Play in Online Games Journal of CyberPsychology and Behavior 9 772 775 2007 17201605
- Rosato A CASD-NMR: critical assessment of automated structure determination by NMR Nature Methods 6 625 626 2009 URL 10.1038/nmeth0909-625 19718014 PMC2841015 · DOI
Cited by
Sorted by citations
The benefits of playing video games.
Granic, I., Lobel, A., & Engels, R. C. M. E. (2014) · American Psychologist · doi:10.1037/a0034857 · Cited by 2,456 · see citing works
Leaver-Fay (2011) · doi:10.1016/b978-0-12-381270-4.00019-6 · Cited by 1,904 · see citing works
The Protein-Folding Problem, 50 Years On
Dill, K. A., & MacCallum, J. L. (2012) · Science · doi:10.1126/science.1219021 · Cited by 1,699 · see citing works
Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441923 · Cited by 1,034 · see citing works
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950 · see citing works
Macromolecular modeling and design in Rosetta: recent methods and frameworks
Leman, J. K., Weitzner, B. D., Lewis, S. M., Adolf-Bryfogle, J., Alam, N., Alford, R. F., Aprahamian, M., Baker, D., Barlow, K. A., Barth, P., et al. (2020) · Nature Methods · doi:10.1038/s41592-020-0848-2 · Cited by 888 · see citing works
Self-citationInteractive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906 · see citing works
Principles for designing ideal protein structures
Koga, N., Tatsumi-Koga, R., Liu, G., Xiao, R., Acton, T. B., Montelione, G. T., & Baker, D. (2012) · Nature · doi:10.1038/nature11600 · Cited by 564 · see citing works
Self-citationSurrogate Wnt agonists that phenocopy canonical Wnt and β-catenin signalling
Janda, C. Y., Dang, L. T., You, C., Chang, J., de Lau, W., Zhong, Z. A., Yan, K. S., Marecic, O., Siepe, D., Li, X., et al. (2017) · Nature · doi:10.1038/nature22306 · Cited by 403 · see citing works
Self-citationFerreiro, D. U., Komives, E. A., & Wolynes, P. G. (2014) · Quarterly Reviews of Biophysics · doi:10.1017/s0033583514000092 · Cited by 314 · see citing works
Locally noisy autonomous agents improve global human coordination in network experiments
Shirado, H., & Christakis, N. A. (2017) · Nature · doi:10.1038/nature22332 · Cited by 288 · see citing works
From Machine Learning to Explainable AI
Holzinger, A. (2018) · 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) · doi:10.1109/disa.2018.8490530 · Cited by 306 · see citing works
Deep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227 · see citing works
Mapping of Crowdsourcing in Health: Systematic Review
Créquit, P., Mansouri, G., Benchoufi, M., Vivot, A., & Ravaud, P. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9330 · Cited by 135 · see citing works
Principles for computational design of binding antibodies
Baran, D., Pszolla, M. G., Lapidoth, G. D., Norn, C., Dym, O., Unger, T., Albeck, S., Tyka, M. D., & Fleishman, S. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1707171114 · Cited by 136 · see citing works
Procko, E., Berguig, G. Y., Shen, B. W., Song, Y., Frayo, S., Convertine, A. J., Margineantu, D., Booth, G., Correia, B. E., Cheng, Y., et al. (2014) · Cell · doi:10.1016/j.cell.2014.04.034 · Cited by 127 · see citing works
Self-citationStrauch, E.-M., Bernard, S. M., La, D., Bohn, A. J., Lee, P. S., Anderson, C. E., Nieusma, T., Holstein, C. A., Garcia, N. K., Hooper, K. A., et al. (2017) · Nature Biotechnology · doi:10.1038/nbt.3907 · Cited by 119 · see citing works
Self-citationBrinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132 · see citing works
Working Together Apart: Collaboration over the Internet
Olson, J. S., & Olson, G. M. (2014) · Synthesis Lectures on Human-Centered Informatics · doi:10.2200/s00542ed1v01y201310hci020 · Cited by 95 · see citing works
Determining crystal structures through crowdsourcing and coursework
Horowitz, S., Koepnick, B., Martin, R., Tymieniecki, A., Winburn, A. A., Cooper, S., Flatten, J., Rogawski, D. S., Koropatkin, N. M., Hailu, T. T., et al. (2016) · Nature Communications · doi:10.1038/ncomms12549 · Cited by 75 · see citing works
Self-citationProtein design via deep learning
Ding, W., Nakai, K., & Gong, H. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac102 · Cited by 66 · see citing works
Utility of human-computer interactions: toward a science of preference measurement
Toomim, M., Kriplean, T., Pörtner, C., & Landay, J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979277 · Cited by 64 · see citing works
Carlson, J., & Harris, K. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000860 · Cited by 61 · see citing works
Roll, I., Russell, D. M., & Gašević, D. (2018) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-018-0170-7 · Cited by 49 · see citing works
Improving microbial phylogeny with citizen science within a mass-market video game
Sarrazin-Gendron, R., Ghasemloo Gheidari, P., Butyaev, A., Keding, T., Cai, E., Zheng, J., Mutalova, R., Mounthanyvong, J., Zhu, Y., Nazarova, E., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-024-02175-6 · Cited by 9 · see citing works
An Ethics Framework for Evaluating Ownership Practices in Biomedical Citizen Science
Guerrini, C. J., & McGuire, A. L. (2022) · Citizen Science: Theory and Practice · doi:10.5334/cstp.537 · Cited by 7 · see citing works
Playing the System: Can Puzzle Players Teach us How to Solve Hard Problems?
Mutalova, R., Sarrazin-Gendron, R., Cai, E., Richard, G., Ghasemloo Gheidari, P., Caisse, S., Knight, R., Blanchette, M., Szantner, A., & Waldispühl, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581375 · Cited by 6 · see citing works
- Authors
- Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F.
- Affiliations
- University of Washington (US)
- Carnegie Mellon University (US)
- Howard Hughes Medical Institute (US)
- Corresponding author
- Baker, David
- Popović, Zoran
- Venue
- Nature
- Published
- 2010-08
- Volume
- 466
- Issue
- 7307
- Pages
- 756-760
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC2956414
- OpenAlex
- W2073338313
- Cited by
- 1,368
- References
- 0
- Citations by year
- 2026: 35 · 2025: 35 · 2024: 59 · 2023: 52 · 2022: 44 · 2021: 79 · 2020: 65 · 2019: 119 · 2018: 87 · 2017: 101 · 2016: 128 · 2015: 123 · 2014: 135 · 2013: 133 · 2012: 95
- FWCI
- 39.76
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Howard Hughes Medical Institute
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
- Keywords
- protein structure prediction, Foldit, online multiplayer games, crowdsourcing, Rosetta, protein folding, search strategies, backbone rearrangement
- MeSH terms
- Algorithms, Computer Graphics, Computer Simulation, Cooperative Behavior, Cues, Games, Experimental, Group Processes, Humans, Hydrogen Bonding, Leisure Activities, Models, Molecular, Photic Stimulation, Problem Solving, Protein Conformation, Proteins, Stochastic Processes, Thermodynamics, Protein Folding, Computational Biology, Nuclear Magnetic Resonance, Biomolecular, Internet, Imaging, Three-Dimensional, Hydrophobic and Hydrophilic Interactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
