Unsupervised machine learning in atomistic simulations, between predictions and understanding

In the paper

“In other terms Eq. (1) reproduces exactly the value of the kernel for each structure in the reference set, and provides a practical approximation of the vector |X〉 associated with the kernel for any other structure.”

In the paper

“Obviously, opting for a specialized description of a system introduces some biases in the procedure.”

In the paper

“From a formal point of view, the representer theorem guarantees that each positive-definite kernel 14 is associated with a Hilbert space whose scalar product 〈X|X ′ 〉 equals the kernel, and it is always possible to convert a kernel into a metric, e.g. by the relation d ( X , X ′ ) 2 = k ( X , X ) + k ( X ′ , X ′ ) -2 k ( X , X ′ ).”

What the sources say

• Not checked yetcites [14] M. Cuturi , e-print arXiv:0911.5367 (2009).

In the paper

“It is worth stressing that the representation of a structure can take the form of a vector of features |X〉 (e.g. all the backbone dihedrals in a protein, the coordination number of atoms in a cluster, etc.), a measure of similarity between pairs of structures that can be implemented as a distance d ( X , X ′ ), or the form of a kernel k ( X , X ′ ). In fact, these three ways of representing a data”

In the paper

“Given a representation in terms of a finite set of features x j ( X ) = 〈 j |X〉 , it is always possible to use it as the starting point to define a distance or a kernel.”

In the paper

“Such a unifying view of input representations helps to understand the limits and strengths of each particular representation - for instance how SOAP power spectra components are systematic but overdetermined, and how Behler-Parrinello symmetry functions are highly correlated with each other.”

In the paper

“It is however hard if not impossible to comprehend - not to mention visualize - the meaning or the relative positions of vectors in a high (or infinite) dimensional space.”

In the paper

“The problem of obtaining a complete yet concise description of a molecular or condensed-phase structure is central to many modelling techniques, including the sampling of rare events 15 and the search for stable structures 2,16,17 .”

What the sources say

• Not checked yetcites [15] 2002, Proc. Natl. Acad. Sci. U. S. A. source
• Not checked yetcites [2] 2011, J. Phys.: Condens. Matter source
• Not checked yetcites [16] 2003
• Not checked yetcites [17] 2006, J. Chem. Phys. source

In the paper

“One may argue that given sufficient amounts of training data a good ML scheme should be able to recognize these invariances, but why bother: most of the state-ofthe-art representations incorporate symmetries and by using them you can let your 20-layer convolutional neural network focus on learning correlations that are not as obvious.”

In the paper

“Clustering techniques identify agglomeration of data in the most significant regions of feature space, but do not offer an overall picture of the relations between different structures.”

In the paper

“In the context of unsupervised learning it is often performed manually, by trial and error, and constitutes one of the critical points to consider to keep the procedure truly unbiased.”

In the paper

“This is a problem that obviously transcends the specific application to chemistry, physics or materials science, and dimensionality-reduction methods have long been sought after for all sorts of data analysis applications.”

In the paper

“Obviously, the separation is far from clear-cut, and it is often possible to construct a local version of a global representation 25 , and vice versa to assemble local kernels and feature vectors into a representation of entire structures 32 .”

What the sources say

• Not checked yetcites [25] 2013, J. Chem. Phys. source
• Not checked yetcites [32] 2016, Phys. Chem. Chem. Phys. source

In the paper

“The use of local, atom/fragment-based representations is also convenient to find recurring patterns in the building blocks of a structure, rather than in the overall configuration.”

In the paper

“As we will discuss below, the risk of introducing spurious features can be mitigated by using general-purpose, relatively simple functions of the atomic coordinates as the input representation, and by checking that clustering is insensitive to the choice feature space, kernel or metric.”

In the paper

“In principle, the information on the absolute position of each input in space, and the relations between any given pair of inputs, are encoded fully in the feature vectors.”

In the paper

“Non-linear transformations of the input space, or of the measure of similarity, introduce distortions that can easily split, merge, create and destroy local maxima in the probability distribution associated with the input set.”

In the paper

“dimensionality reduction lies at the heart of the vast majority of accelerated sampling methods, which try to beat the time scale problem of molecular simulations”

In the paper

“The idea that the data may lie on a manifold that is only low-dimensional on a local basis is also behind methods such as locally-linear embedding (LLE) 76”

What the sources say

• Not checked yetcites [76] 2000, Science source

In the paper

“The time-lagged independent component analysis method (TICA) 80,81 identifies linear combinations of features from a time-dependent trajectory that not only have the largest variance, but that also have the longest autocorrelation time”

What the sources say

• Not checked yetcites [80] 1994, Phys. Rev. Lett. source
• Not checked yetcites [81] 2013, J. Chem. Phys. source

In the paper

“the spectral gap optimization of order parameters (SGOOP) method to single out the linear combination of input features that correspond to the direction that is most relevant for the long-time dynamics of the system being studied 82”

What the sources say

• Not checked yetcites [82] 2016, Proc. Natl. Acad. Sci. U. S. A. source

In the paper

“'Materials cartography', as it has been referred to 57 , strives to obtain a low-dimensional (typically 2D) representation of a set of materials, or of different phases of the same material, that reflects the 'structureenergy-property' landscape 84,85 of that system.”

What the sources say

• Not checked yetcites [57] 2015, Chem. Mater. source
• Not checked yetcites [84] 2018, Chem. Sci. source
• Not checked yetcites [85] 2018, Chem. Mater. source

In the paper

“Similar approaches have been applied to biomolecular simulations 86 , where the main application of dimensionality reduction has been in connection with accelerated sampling.”

What the sources say

• Not checked yetcites [86] 2012, J. Mol. Graphics Modell. source

In the paper

“All of the methods discussed this far rely exclusively on the distribution of atoms in each structure.”

In the paper

“ISOMAP 75 uses a neighborhood analysis to estimate geodesic distances between points, based on the assumption that the data lies on a curved, but locally low-dimensional, manifold.”

What the sources say

• Not checked yetcites [75] 2000, Science source

In the paper

“The applications of dimensionality reduction schemes to atomistic simulations reflect the dual function of a coarse-grained representation, as a mean to simplify the description of a system to understand it better, and to explore more effectively its conformational space.”

In the paper

“In order to observe a rare (activated) event without developing dedicated hardware 1 one needs to selectively increase the speed at which the pathways related to the transition are traversed 51,87,88 , to modify the probability distribution that is sampled during the simulation so that configurations that are close to the transition state occur more often 15,89 , or to generate adaptively ens”

What the sources say

• Not checked yetcites [1] 2009
• Not checked yetcites [51] 2002, Annu. Rev. Mater. Res. source
• Not checked yetcites [87] 2002, Annu. Rev. Phys. Chem. source
• Not checked yetcites [88] 2008, J. Phys. Chem. B source
• Not checked yetcites [15] 2002, Proc. Natl. Acad. Sci. U. S. A. source
• Not checked yetcites [89] 1977, J. Comput. Phys. source

In the paper

“Take for instance the case of metadynamics, in which a repulsive bias is accumulated in locations that have already been visited during the trajectory, forcing the system out of free energy minima 93,94 . The volume of one of these minima grows exponentially with the dimensionality d of collective variable space, and so does the time to accumulate sufficient bias to escape the freeenergy basin.”

What the sources say

• Not checked yetcites [93] 2005, J. Phys. Chem. B source
• Not checked yetcites [94] 2008, Rep. Prog. Phys. source

In the paper

“Either way, the objective is to accelerate sampling while introducing a minimal disturbance to the natural behavior of the system 91”

What the sources say

• Not checked yetcites [91] 2013, Phys. Rev. Lett. source

In the paper

“The accuracy of the regression model also provides an objective metric to optimize the hyperparameters that define the representation 107 , and even to determine the most effective non-linear function of features that reproduces the target properties 108 .”

What the sources say

• Not checked yetcites [107] 2018, Phys. Chem. Chem. Phys. source
• Not checked yetcites [108] 2018, Phys. Rev. Mater. source

In the paper

“It is then tempting to pick one own's arbitrary perception of what an 'insightful' outcome of the analysis would be, and tune the representation and machine-learning scheme until the results reflect most clearly one's expectations.”

In the paper

“For starters, it gives an idea of the limiting accuracy that can be expected when inferring structureproperty relations by application of unsupervised classification or coarse-graining schemes.”

In the paper

“Clustering can be used to select the most relevant reference configurations from a database 109 , or to partition it into regions on which specialized models can be trained. 110”

What the sources say

• Not checked yetcites [109] 2015, Int. J. Quantum Chem. source
• Not checked yetcites [110] 2018, J. Chem. Theory Comput. source

In the paper

“A low-dimensional representation of a dataset can help when it comes to identifying problematic regions, and understanding how a given representation 'sees' a set of structures 58,104,111 .”

What the sources say

• Not checked yetcites [58] 2017, J. Cheminf. source
• Not checked yetcites [104] 2017, Sci. Adv. source
• Not checked yetcites [111] 2018, J. Chem. Phys. source

In the paper

“Choosing to describe the inputs using a feature vector that is not problem specific, and validating and optimizing it through a relevant supervised learning exercise, reduces substantially the amount of human intervention and the possibility of introducing preconceived biases when applying clustering and dimensionality reduction schemes.”

In the paper

“Unsupervised schemes can help shed light on the functioning of supervised algorithms that are often opaque, and supervised schemes can help to formulate quantitative metrics to assess the performance of unsupervised learning.”