Playing the System: Can Puzzle Players Teach us How to Solve Hard Problems?
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
With nearly three billion players, video games are more popular than ever. Casual puzzle games are among the most played categories. These games capitalize on the players’ analytical and problem-solving skills. Can we leverage these abilities to teach ourselves how to solve complex combinatorial problems? In this study, we harness the collective wisdom of millions of players to tackle the classical NP-hard problem of multiple sequence alignment, relevant to many areas of biology and medicine. We show that Borderlands Science players propose solutions to multiple sequence alignment tasks that perform as well or better than standard approaches, while exploring a much larger area of the Pareto-optimal solution space. We also show the strategies of the players, although highly heterogeneous, follow a collective logic that can be mimicked with Behavioral Cloning with minimal performance loss, allowing the players’ collective wisdom to be leveraged for alignment of any sequences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In the first study52, we established that it is possible for reinforcement learning agents to learn player strategies for solving BLS puzzles”
From Improving microbial phylogeny with citizen science within a mass-market video game (2025) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Needleman, S. B., & Wunsch, C. D. (1970) · Journal of Molecular Biology · doi:10.1016/0022-2836(70)90057-4 · Cited by 11,288
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680 · cited 1×
“The most novel component of the encoder section is the Multihead attention layer, a special layer that allows each input vector to interact with other words through the attention mechanism, instead of passing through a hidden state like in an RNN [Sherstinsky 2020] or CNN [LeCun et al. 1989].”
BackgroundA Greedy Algorithm for Aligning DNA Sequences
Zhang, Z., Schwartz, S., Wagner, L., & Miller, W. (2000) · Journal of Computational Biology · doi:10.1089/10665270050081478 · Cited by 5,478
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · cited 1×
“…of alternative strategies for future work, as recent work has confrmed it is possible to train an agent to play at an experienced level in a game with behavioural cloning [Vinyals et al. 2019], and this is the extent of the evidence we needed to accumulate for the claims presented in this work.”
BackgroundLabeling images with a computer game
von Ahn, L., & Dabbish, L. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985733 · Cited by 2,204
Aligning Multiple Genomic Sequences With the Threaded Blockset Aligner
Blanchette, M., Kent, W. J., Riemer, C., Elnitski, L., Smit, A. F. A., Roskin, K. M., Baertsch, R., Rosenbloom, K., Clawson, H., Green, E. D., et al. (2004) · Genome Research · doi:10.1101/gr.1933104 · Cited by 1,452
Self-citationPredicting protein structures with a multiplayer online game
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010) · Nature · doi:10.1038/nature09304 · Cited by 1,368 · cited 4×
“Gamifcation of scientifc tasks has gained traction since the release of Foldit in 2008 [Cooper et al. 2010], with many science discovery games reaching hundreds of thousands of players, such as EteRNA [Lee et al.”
BackgroundInfluentialGapped sequence alignment using artificial neural networks: application to the MHC class I system
Andreatta, M., & Nielsen, M. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btv639 · Cited by 1,087
Imitation Learning: A Survey of Learning Methods
Hussein, A., Gaber, M. M., Elyan, E., & Jayne, C. (2018) · ACM Computing Surveys · doi:10.1145/3054912 · Cited by 1,060 · cited 2×
“AI Players based on the players behavior can solve a large number of diferent tasks [Hussein et al. 2017].”
BackgroundMethodWeakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation
Papandreou, G., Chen, L.-C., Murphy, K. P., & Yuille, A. L. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.203 · Cited by 1,005
von Ahn, L. (2006) · Computer · doi:10.1109/mc.2006.196 · Cited by 955
On the Complexity of Multiple Sequence Alignment
WANG, L., & JIANG, T. (1994) · Journal of Computational Biology · doi:10.1089/cmb.1994.1.337 · Cited by 934
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889
American Gut: an Open Platform for Citizen-Science Microbiome Research
McDonalda, D., Hyde, E., Debelius, J. W., Morton, J. T., Gonzalez, A., Ackermann, G., Aksenov, A. A., Behsaz, B., Brennan, C., Chen, Y., et al. (2018) · doi:10.1101/277970 · Cited by 878
Self-citationJensen-Shannon divergence and Hilbert space embedding
Fuglede, B., & Topsoe, F. (2004) · International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings. · doi:10.1109/isit.2004.1365067 · Cited by 612
Learning similarity with cosine similarity ensemble
Xia, P., Zhang, L., & Li, F. (2015) · Information Sciences · doi:10.1016/j.ins.2015.02.024 · Cited by 552
Lu, X., Li, B., Yue, Y., Li, Q., & Yan, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00754 · Cited by 533
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Codevilla, F., Santana, E., Lopez, A., & Gaidon, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00942 · Cited by 509
PASTA: Ultra-Large Multiple Sequence Alignment for Nucleotide and Amino-Acid Sequences
Mirarab, S., Nguyen, N., Guo, S., Wang, L.-S., Kim, J., & Warnow, T. (2015) · Journal of Computational Biology · doi:10.1089/cmb.2014.0156 · Cited by 420
Axial Attention in Multidimensional Transformers
Ho, J., Kalchbrenner, N., Weissenborn, D., & Salimans, T. (2019) · arXiv · doi:10.48550/arxiv.1912.12180 · Cited by 370
A framework for behavioural cloning
Bain, M., & Sammut, C. (2000) · Machine Intelligence 15 · doi:10.1093/oso/9780198538677.003.0006 · Cited by 360
Distance Weighted Cosine Similarity Measure for Text Classification
Li, B., & Han, L. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-41278-3_74 · Cited by 321
RNA design rules from a massive open laboratory
Lee, J., Kladwang, W., Lee, M., Cantu, D., Azizyan, M., Kim, H., Limpaecher, A., Gaikwad, S., Yoon, S., Treuille, A., et al. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313039111 · Cited by 301
Analysis of symbolic sequences using the Jensen-Shannon divergence
Grosse, I., Bernaola-Galván, P., Carpena, P., Román-Roldán, R., Oliver, J., & Stanley, H. E. (2002) · Physical Review E · doi:10.1103/physreve.65.041905 · Cited by 297
Vehicle Detection from 3D Lidar Using Fully Convolutional NetworkPreprint
Li, B., Zhang, T., & Xia, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07916 · Cited by 275
Sequence alignment and penalty choice
Vingron, M., & Waterman, M. S. (1994) · Journal of Molecular Biology · doi:10.1016/s0022-2836(05)80006-3 · Cited by 249
A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles
Le Mero, L., Yi, D., Dianati, M., & Mouzakitis, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3144867 · Cited by 243
A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion
Song, S., & Geyer, H. (2015) · The Journal of Physiology · doi:10.1113/jp270228 · Cited by 242
Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment
Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., Zarour, E., players, P., Sarmenta, L., Blanchette, M., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031362 · Cited by 234
Self-citationMultiple sequence alignment modeling: Methods and applications
Chatzou, M., Magis, C., Chang, J.-M., Kemena, C., Bussotti, G., Erb, I., & Notredame, C. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbv099 · Cited by 229
Attention-Aware Deep Reinforcement Learning for Video Face Recognition
Rao, Y., Lu, J., & Zhou, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.424 · Cited by 182
MineRL: A Large-Scale Dataset of Minecraft DemonstrationsPreprint
Guss, W. H., Houghton, B., Topin, N., Wang, P., Codel, C. R., Veloso, M., & Salakhutdinov, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13440 · Cited by 170
Scoring profile‐to‐profile sequence alignments
Wang, G., & Dunbrack, R. L. (2004) · Protein Science · doi:10.1110/ps.03601504 · Cited by 133
Evolution of 5S RNA and the Non-randomness of Base Replacement
Sankoff, D., MOREL, C., & Cedergren, R. (1973) · Nature New Biology · doi:10.1038/newbio245232a0 · Cited by 104
Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
Zhang, R., Walshe, C., Liu, Z., Guan, L., Muller, K., Whritner, J. A., Zhang, L., Hayhoe, M., & Ballard, D. H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6161 · Cited by 88
Some Video Games Can Increase the Player's Creativity
Moffat, D. C., Crombie, W., & Аркадьевна, Ш. О. (2017) · International Journal of Game-Based Learning · doi:10.4018/ijgbl.2017040103 · Cited by 61
BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement LearningPreprint
Chen, X., Zhou, Z., Wang, Z., Wang, C., Wu, Y., & Ross, K. W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12179 · Cited by 59
The effects of expected reward on creative problem solving
Cristofori, I., Salvi, C., Beeman, M. J., & Grafman, J. H. (2018) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-018-0613-5 · Cited by 51
Wisdom of artificial crowds algorithm for solving NP-hard problems
Yampolskiy, R. V., & El-Barkouky, A. (2011) · International Journal of Bio-Inspired Computation · doi:10.1504/ijbic.2011.043624 · Cited by 48
Dungeons & Replicants: Automated Game Balancing via Deep Player Behavior Modeling
Pfau, J., Liapis, A., Volkmar, G., Yannakakis, G. N., & Malaka, R. (2020) · 2020 IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231958 · Cited by 48
Using deep reinforcement learning approach for solving the multiple sequence alignment problem
Jafari, R., Javidi, M. M., & Rafsanjani, M. K. (2019) · SN Applied Sciences · doi:10.1007/s42452-019-0611-4 · Cited by 41
Waldispühl, J., Szantner, A., Knight, R. D., Caisse, S., & Pitchford, R. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0694-x · Cited by 38
Self-citationMultiple sequence alignment with user-defined constraints at GOBICS
Morgenstern, B., Werner, N., Prohaska, S. J., Steinkamp, R., Schneider, I., Subramanian, A. R., Stadler, P. F., & Weyer-Menkhoff, J. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bti142 · Cited by 32
Benchmarking End-to-End Behavioural Cloning on Video Games
Kanervisto, A., Pussinen, J., & Hautamäki, V. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231600 · Cited by 31
Gut Instinct: Creating Scientific Theories with Online Learners
Pandey, V., Amir, A., Debelius, J. W., Hyde, E. R., Kościółek, T., Knight, R. D., & Klemmer, S. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025769 · Cited by 30
Self-citationCloning Safe Driving Behavior for Self-Driving Cars using Convolutional Neural Networks
Farag, W. (2018) · Recent Advances in Computer Science and Communications · doi:10.2174/2213275911666181106160002 · Cited by 28
Sammut, C. (2010) · Springer eBooks · doi:10.1007/978-0-387-30164-8_69 · Cited by 27
Acuna, D. E., & Parada, V. (2010) · PLoS ONE · doi:10.1371/journal.pone.0011685 · Cited by 20
Galileo: Citizen-led Experimentation Using a Social Computing System
Pandey, V., Koul, T., Yang, C., McDonald, D. T., Ball, M. P., Tzovaras, B. G., Knight, R. D., & Klemmer, S. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445668 · Cited by 14
Self-citationDLPAlign: A Deep Learning based Progressive Alignment Method for Multiple Protein Sequences
Kuang, M., Liu, Y., & Gao, L. (2020) · International Conference Computational Systems-Biology and Bioinformatics · doi:10.1145/3429210.3429221 · Cited by 9
Salama, R. A., & Stekel, D. J. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt463 · Cited by 7
Behavioral Cloning in Atari Games Using a Combined Variational Autoencoder and Predictor Model
Chen, B., Tandon, S., Gorsich, D., Gorodetsky, A., & Veerapaneni, S. (2021) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec45853.2021.9505001 · Cited by 6
Can You Rely on Human Computation?
Pfau, J., & Malaka, R. (2019) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3341215.3356297 · Cited by 2
Levelling Up: The Cultural Impact of Contemporary Videogames
(2016) · BRILL eBooks · doi:10.1163/9781848884380 · Cited by 2
Human-supervised clustering of multidimensional data using crowdsourcing
Butyaev, A., Drogaris, C., Tremblay-Savard, O., & Waldispühl, J. (2022) · Royal Society Open Science · doi:10.1098/rsos.211189 · Cited by 2
Self-citationComparing Problem Solving Strategies for NP-hard Optimization Problems
Hidalgo-HerreroMercedes, RabanalPablo, RodríguezIsmael, & RubioFernando (2013) · Fundamenta Informaticae · doi:10.5555/2594895.2594897 · Cited by 1
Research works printed but not linked: 27
- Shreyansh Daftry, J Andrew Bagnell, and Martial Hebert. 2016. Learning transferable policies for monocular reactive mav control. In International Symposium on Experimental Robotics. Springer, 3–11.
- Chuong B Do, Mahathi SP Mahabhashyam, Michael Brudno, and Serafim Batzoglou. 2005. ProbCons: Probabilistic consistency-based multiple sequence alignment. Genome research 15, 2 (2005), 330–340.
- Sean Eddy. 1992. HMMER user’s guide. Department of Genetics, Washington University School of Medicine 2, 1 (1992), 13.
- Sean Eddy. 2022. 2022 Essential Facts About the Video Game Industry. Entertainment Software Association (2022).
- Christopher B Eiben, Justin B Siegel, Jacob B Bale, Seth Cooper, Firas Khatib, Betty W Shen, Barry L Stoddard, Zoran Popovic, and David Baker. 2012. Increased Diels-Alderase activity through backbone remodeling guided by Foldit players. Nature biotechnology 30, 2 (2012), 190–192.
- Reza Jafari Hamed Tabrizchi Marjan Kuchaki Rafsanjani and M.M. Javidi. 2020. A Distributed Deep Q-learning based Model for Solving the Multiple Sequence Alignment Problem. · DOI
- Matthias J Koepp, Roger N Gunn, Andrew D Lawrence, Vincent J Cunningham, Alain Dagher, Tasmin Jones, David J Brooks, Christopher J Bench, and PM Grasby. 1998. Evidence for striatal dopamine release during a video game. Nature 393, 6682 (1998), 266–268.
- Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. 1989. Backpropagation applied to handwritten zip code recognition. Neural computation 1, 4 (1989), 541–551.
- Hong-Liang Li, Yi-He Pang, and Bin Liu. 2021. BioSeq-BLM: a platform for analyzing DNA, RNA and protein sequences based on biological language models. Nucleic acids research 49, 22 (2021), e129–e129.
- Ari Löytynoja and Nick Goldman. 2008. Phylogeny-aware gap placement prevents errors in sequence alignment and evolutionary analysis. science 320, 5883 (2008), 1632–1635.
- Umberto Lupo, Damiano Sgarbossa, and Anne-Florence Bitbol. 2022. Protein language models trained on multiple sequence alignments learn phylogenetic relationships. bioRxiv (2022).
- James N MacGregor and Tom Ormerod. 1996. Human performance on the traveling salesman problem. Perception & psychophysics 58, 4 (1996), 527–539.
- 10.1128/mSystems.00031-18 · doi:10.1128/msystems.00031-18 · Paper page
- Hieu V Nguyen and Li Bai. 2010. Cosine similarity metric learning for face verification. In Asian conference on computer vision. Springer, 709–720.
- Nam-phuong D Nguyen, Siavash Mirarab, Keerthana Kumar, and Tandy Warnow. 2015. Ultra-large alignments using phylogeny-aware profiles. Genome biology 16, 1 (2015), 1–15.
- Mohammad Norouzi David J Fleet and Russ R Salakhutdinov. 2012. Hamming distance metric learning. In Advances in neural information processing systems. 1061–1069.
- Benedict Paten, Javier Herrero, Kathryn Beal, Stephen Fitzgerald, and Ewan Birney. 2008. Enredo and Pecan: genome-wide mammalian consistency-based multiple alignment with paralogs. Genome research 18, 11 (2008), 1814–1828.
- Aluísio Pinheiro, Hildete Prisco Pinheiro, and Pranab Kumar Sen. 2012. The use of Hamming distance in bioinformatics. In Handbook of Statistics. Vol. 28. Elsevier, 129–162.
- Dean A Pomerleau. 1989. Alvinn: An autonomous land vehicle in a neural network. Technical Report. CARNEGIE-MELLON UNIV PITTSBURGH PA ARTIFICIAL INTELLIGENCE AND PSYCHOLOGY ….
- Ramchalam Kinattinkara Ramakrishnan, Jaspal Singh, and Mathieu Blanchette. 2018. Rlalign: a reinforcement learning approach for multiple sequence alignment. In 2018 IEEE 18th International Conference on Bioinformatics and Bioengineering (BIBE). IEEE, 61–66.
- Roshan Rao, Joshua Meier, Tom Sercu, Sergey Ovchinnikov, and Alexander Rives. 2020. Transformer protein language models are unsupervised structure learners. Biorxiv (2020).
- J Redding, Jacob Schreiver, C Shrum, A Lauf, and R Yampolskiy. 2015. Solving NP-hard number matrix games with Wisdom of Artificial Crowds. In 2015 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES). IEEE, 38–43.
- Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 627–635.
- Julie D Thompson, Desmond G Higgins, and Toby J Gibson. 1994. CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. Nucleic acids research 22, 22 (1994), 4673–4680.
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008.
- 10.1145/985692.985733 · DOI · Paper page
- Sheng Kung Michael Yi, Mark Steyvers, Michael D Lee, and Matthew J Dry. 2012. The wisdom of the crowd in combinatorial problems. Cognitive science 36, 3 (2012), 452–470.
Cited by
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Improving microbial phylogeny with citizen science within a mass-market video game
Sarrazin-Gendron, R., Ghasemloo Gheidari, P., Butyaev, A., Keding, T., Cai, E., Zheng, J., Mutalova, R., Mounthanyvong, J., Zhu, Y., Nazarova, E., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-024-02175-6 · Cited by 9
Self-citationProceedings of The ACM Collective Intelligence Conference
(2023) · doi:10.1145/3582269 · Cited by 4
Leadership Through Citizen Science Games
Schrier, K. “. (2024) · Journal of Leadership Studies · doi:10.1002/jls.21878 · Cited by 2
Player-Guided AI outperforms standard AI in Sequence Alignment Puzzles
Mutalova, R., Sarrazin-Gendron, R., Ghasemloo Gheidari, P., Cai, E., Richard, G., Caisse, S., Knight, R., Blanchette, M., Szantner, A., & Waldispühl, J. (2023) · Proceedings of The ACM Collective Intelligence Conference · doi:10.1145/3582269.3615597 · Cited by 1
Self-citation
Metadata
- Authors
- Mutalova, R., Sarrazin-Gendron, R., Cai, E., Richard, G., Ghasemloo Gheidari, P., Caisse, S., Knight, R., Blanchette, M., Szantner, A., & Waldispühl, J.
- Affiliations
- McGill University (CA)
- School of Computer Science, McGill University, Canada
- Gearbox Studio Québec, Canada
- University of California San Diego (US)
- Massively Multiplayer Online Science, Switzerland
- ORCID
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-15
- OpenAlex
- W4366590983
- Cited by
- 6
- References
- 56
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 2
- FWCI
- 6.40
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nc-sa)
- Funders
- McGill University
- Genome Canada
- Génome Québec
- University of California, San Diego
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- puzzle games, multiple sequence alignment, NP-hard problems, combinatorial optimization, collective intelligence, crowdsourcing, behavior cloning, Pareto optimal solutions, player strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.