Determining crystal structures through crowdsourcing and coursework

According to Molprobity, the top Foldit structure is an exceptional model, ranking in the 100th percentile in both its overall Molprobity score and clashscore of all structures in the PDB of similar resolution (1.95±0.25 Å).

“According to Molprobity13, the top Foldit structure is an exceptional model, ranking in the 100th percentile in both its overall Molprobity score and clashscore of all structures in the PDB of similar resolution (1.95±0.25 Å).”

• cites [13] Chen, V. B. et al. MolProbity: all-atom structure validation for macromolecular crystallography. Acta Crystallogr. D Biol. Crystallogr. 66, 12–21 (2010). sourceSupported

• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

In a class assignment, about 10% of 57 undergraduate students improved on the previously published model of lectin scytovirin.

“In addition to learning about protein structure, ∼10% of these students improved on the previously published model4.”

• cites [4] Horowitz, S., Koldewey, P. & Bardwell, J. C. Undergraduates improve upon published crystal structure in class assignment. Biochem. Mol. Biol. Educ. 42, 398–404 (2014). sourceSupported

About 85% of deposited protein crystal structures contain discernable errors.

“As a result, ∼85% of deposited protein crystal structures contain discernable errors2.”

• cites [2] Touw, W. G., Joosten, R. P. & Vriend, G. New biological insights from better structure models. J Mol Biol 428, 1375–1393 (2016). sourcePartly supported

  The Validation section describes errors found in deposited crystal structures and methods for detecting them. The Concluding remarks also say X-ray structures may contain experimental and human errors. Neither passage, nor the passages retrieved by searching for 85% and error prevalence, supports the claim that approximately 85% of deposited protein crystal structures contain discernable errors (pages unavailable in our copy). (LLM, conf=0.84)

  Experimental — accuracy not yet measured

As a preliminary test, Foldit players were able to improve on the published scytovirin structure.

“As a preliminary test of this feature, we gave an electron density puzzle to the Foldit players that was nearly identical to the lectin scytovirin classroom assignment mentioned above, and found that the Foldit players were also able to improve on the published scytovirin structure (Supplementary Note 1; Supplementary Fig. 2).”

• cites [4] Horowitz, S., Koldewey, P. & Bardwell, J. C. Undergraduates improve upon published crystal structure in class assignment. Biochem. Mol. Biol. Educ. 42, 398–404 (2014). sourceSupported

A DALI search of YPL067C found that it is structurally similar to members of the widely conserved superfamily of histidine triad (HIT) proteins.

“Despite the lack of sequence homology to any protein in the PDB, a DALI16 search of YPL067C (Supplementary Table 1) found that it is structurally similar to members of the widely conserved superfamily of histidine triad (HIT) proteins.”

• cites [16] Holm, L. & Rosenstrom, P. Dali server: conservation mapping in 3D. Nucleic Acids Res. 38, W545–W549 (2010). sourceSupported

ARP/Warp and Phenix Autobuild did not create models of as high quality as Phenix Autosolve or MR-Rosetta.

“ARP/Warp19 and Phenix Autobuild20 did not create models of as high quality as Phenix Autosolve or MR-Rosetta, and were thus not analysed in the competition.”

• cites [19] Langer, G., Cohen, S. X., Lamzin, V. S. & Perrakis, A. Automated macromolecular model building for X-ray crystallography using ARP/wARP version 7. Nat. Protoc. 3, 1171–1179 (2008). sourceSupported
• cites [20] Terwilliger, T. C. et al. Iterative model building, structure refinement and density modification with the PHENIX AutoBuild wizard. Acta Crystallogr. D Biol. Crystallogr. 64, 61–69 (2008). sourceSupported

The top DALI match to the crystal structure of YPL067C was to a HIT protein of unknown function from Clostridium difficile (PDB: 4EGU), with a Z-score of 4.9.

“The top DALI16 match to the crystal structure of YPL067C was to a HIT protein of unknown function from Clostridium difficile (PDB: 4EGU), with a Z-score of 4.9.”

• cites [16] Holm, L. & Rosenstrom, P. Dali server: conservation mapping in 3D. Nucleic Acids Res. 38, W545–W549 (2010). sourceSupported

Introduction of a new feature into Foldit allows players to build and real-space refine structures into electron density maps.

“Introduction of a new feature into the computer game Foldit allows players to build and real-space refine structures into electron density maps.”

YPL067C's structure yielded unexpected insights into its biological function.

“YPL067C's structure yielded unexpected insights into its biological function.”

54% of participating Foldit players had attempted to solve early electron density puzzles in Foldit and thus had some experience in Foldit-based model building.

“In contrast, 54% of the participating Foldit players had attempted to solve early electron density puzzles in Foldit and thus had some experience in Foldit-based model building.”

One student generated a structure that ranked in the 100th percentile in both Molprobity clashscore and total score when compared with other structures in its resolution range.

“One student even generated a structure that ranked in the 100th percentile in both Molprobity clashscore and total score when compared with other structures in its resolution range.”

After removal of disordered residues, a team of Foldit players achieved the most accurate structure in the competition.

“After removal of disordered residues, a team of Foldit players achieved the most accurate structure.”

Better Foldit scores were associated with lower Rfree values.

“Better Foldit scores were associated with lower Rfree values (Supplementary Fig. 6), suggesting that the Foldit model-building strategy and its scoring algorithm could be generalizable as a means of producing high-quality structures.”

The Foldit players' building process sampled conformational space very widely, unlike the Coot users.

“Although the Foldit players also used a large aromatic residue to initially anchor the sequence similar to the Coot users, their building process sampled conformational space very widely (Supplementary Movie 1), unlike the Coot users.”

Analysing the competition target protein YPL067C uncovered a new family of histidine triad proteins apparently involved in the prevention of amyloid toxicity.

“Analysing the target protein of the competition, YPL067C, uncovered a new family of histidine triad proteins apparently involved in the prevention of amyloid toxicity.”

The Foldit structures improved considerably after pruning.

“The Foldit structures improved considerably after pruning.”

• AssumptionRests on an assumption: Disordered atoms do not substantially contribute to X-ray diffraction, so modelling disordered residues will disagree with diffraction data and inflate Rfree values.
• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

The Rfree value of the pruned Foldit structure was marginally better than that achieved by the trained crystallographers.

“In addition to the Rfree value of the pruned Foldit structure becoming marginally better than that achieved by the trained crystallographers and containing zero Ramachandran outliers, the Foldit structure had the lowest level of steric clashes (Fig. 2).”

• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

The new Foldit electron density feature has revealed that non-expert citizen scientists are capable of using structural data to build first-rate models.

“The new Foldit electron density feature has revealed that non-expert citizen scientists are capable of using structural data to build first-rate models, and we expect that Foldit will be a powerful tool for crowdsourcing many new high-quality structures.”

The competition showed that Foldit players could build very high-quality crystal structures into electron density maps.

“The competition showed that Foldit players could build very high-quality crystal structures into electron density maps, opening up a new method for building and refining crystal structures.”

Crystals of YPL067C diffracted to 1.9 Å resolution.

“Crystals of this protein diffracted to 1.9 Å resolution (Table 1).”

The top pruned Foldit structure was the overall highest-quality structure produced in the competition as measured by geometry, density fit and steric clashes.

“As a result, the top pruned Foldit structure was the overall highest-quality structure produced in the competition (Fig. 2) as measured by geometry, density fit and steric clashes.”

• AssumptionRests on an assumption: Disordered atoms do not substantially contribute to X-ray diffraction, so modelling disordered residues will disagree with diffraction data and inflate Rfree values.
• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

The Foldit structure had the lowest level of steric clashes.

“In addition to the Rfree value of the pruned Foldit structure becoming marginally better than that achieved by the trained crystallographers and containing zero Ramachandran outliers, the Foldit structure had the lowest level of steric clashes (Fig. 2).”

• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

Computer game players can build high-quality crystal structures.

“We show here that computer game players can build high-quality crystal structures.”

The top pruned Foldit structure contained zero Ramachandran outliers.

“In addition to the Rfree value of the pruned Foldit structure becoming marginally better than that achieved by the trained crystallographers and containing zero Ramachandran outliers, the Foldit structure had the lowest level of steric clashes (Fig. 2).”

• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

Foldit players can build structural models at least as effectively as trained crystallographers and state-of-the-art automated methods.

“Here we show that Foldit players can build structural models at least as effectively as trained crystallographers and state-of-the-art automated methods, enabling a novel crowd-powered strategy for solving high-accuracy crystal structures.”

Coot users, Phenix Autosolve and MR-Rosetta were able to exclude regions that had poor quality electron density.

“Whereas the Coot users, Phenix Autosolve and MR-Rosetta were able to exclude regions that had poor quality electron density, the current version of Foldit required that players model the entire sequence without gaps, retaining all disordered residues.”

YPL067C is a yeast protein with no significant sequence similarity to any structure in the Protein Data Bank.

“We chose YPL067C, a yeast protein with no significant sequence similarity to any structure in the Protein Data Bank (PDB) (Supplementary Note 2) as the target for our competition.”

HTC1 is very effective in preventing in vitro amyloid formation of three model proteins, Aβ1-40, α-synuclein and reduced carboxy-methylated α-lactalbumin (RCMalA).

“We find here that HTC1 is very effective in preventing in vitro amyloid formation of three model proteins, Aβ1–40, α-synuclein and reduced carboxy-methylated α-lactalbumin (RCMalA; Fig. 4).”

The Foldit players also used a large aromatic residue to initially anchor the sequence similar to the Coot users.

“Although the Foldit players also used a large aromatic residue to initially anchor the sequence similar to the Coot users, their building process sampled conformational space very widely (Supplementary Movie 1), unlike the Coot users.”

Incubation of YPL067C with ThT alone produced no fluorescence increase.

“Incubation of YPL067C with ThT alone produced no fluorescence increase.”

No sequences in the PDB were found for YPL067C.

“No sequences in the PDB were found.”

The flowthrough contained >95% pure YPL067C as measured by SDS-polyacrylamide gel electrophoresis.

“The flowthrough contained >95% pure YPL067C as measured by SDS–polyacrylamide gel electrophoresis.”

YPL067C's most notable distinguishing structural characteristic is an open channel not found in other HIT proteins.

“Its most notable distinguishing structural characteristic is an open channel not found in other HIT proteins (Supplementary Note 2).”

Native YPL067C was solved by molecular replacement with the initial SeMet structure.

“Native YPL067C was solved by molecular replacement with the initial SeMet structure.”

The best structures from Foldit, as measured by Rfree, came from the group of highest-scoring Foldit structures according to Foldit score.

“Notably, the best structures from Foldit, as measured by Rfree, came from the group of highest-scoring Foldit structures according to Foldit score.”

The ease with which students and Foldit players were able to interpret and understand density maps suggests that scientists other than crystallographers can very readily interpret electron density maps.

“Importantly, the ease with which students and Foldit players were able to interpret and understand density maps suggests that scientists other than crystallographers can very readily interpret electron density maps, which will assist them in designing or analysing experiments based on crystal structures.”

The superiority of the top Foldit structure can be attributed to better side-chain conformations than those in the top structure produced by the trained crystallographers.

“The superiority of the top Foldit structure can be attributed to better side-chain conformations than those in the top structure produced by the trained crystallographers (Supplementary Fig. 5).”

• AssumptionRests on an assumption: The top trained crystallographer's model includes only the well-ordered regions of YPL067C, so pruning Foldit models to those residues removes disordered residues.

The Foldit model-building strategy and its scoring algorithm could be generalizable as a means of producing high-quality structures.

“Better Foldit scores were associated with lower Rfree values (Supplementary Fig. 6), suggesting that the Foldit model-building strategy and its scoring algorithm could be generalizable as a means of producing high-quality structures.”

This video game can help produce crystallographic models of higher quality than those from trained crystallographers or automated model-building algorithms alone in at least some cases.

“This surprising win by Foldit players suggests that, in at least some cases, this video game can help produce crystallographic models of higher quality than those from trained crystallographers or automated model-building algorithms alone.”

The difference in accuracy is likely in part the result of different underlying philosophies behind Coot and Foldit.

“The difference in accuracy is likely in part the result of different underlying philosophies behind Coot and Foldit.”

On the basis of docking simulations, HTC1 may bind to unfolded proteins using its conserved channel.

“On the basis of docking simulations, HTC1 may bind to unfolded proteins using its conserved channel (Supplementary Note 2; Supplementary Fig. 8).”

The assignment helped give students a realistic view about the power and limitations of the scientific method.

“This assignment thus helped give students a realistic view about the power and limitations of the scientific method.”

The crowdsourcing-enabled discovery of a new family of proteins involved in preventing amyloid formation provides insight into a novel physiological role of the ubiquitous HIT proteins.

“Our crowdsourcing-enabled discovery of a new family of proteins involved in preventing amyloid formation provides insight into a novel physiological role of the ubiquitous HIT proteins.”