The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations
Open accessAbstract
Recent advances in scanning transmission electron and scanning probe microscopies have opened exciting opportunities in probing the materials structural parameters and various functional properties in real space with angstrom-level precision. This progress has been accompanied by an exponential increase in the size and quality of data sets produced by microscopic and spectroscopic experimental techniques. These developments necessitate adequate methods for extracting relevant physical and chemical information from the large data sets, for which a priori information on the structures of various atomic configurations and lattice defects is limited or absent. Here we demonstrate an application of deep neural networks to extract information from atomically resolved images including location of the atomic species and type of defects. We develop a "weakly supervised" approach that uses information on the coordinates of all atomic species in the image, extracted via a deep neural network, to identify a rich variety of defects that are not part of an initial training set. We further apply our approach to interpret complex atomic and defect transformation, including switching between different coordination of silicon dopants in graphene as a function of time, formation of peculiar silicon dimer with mixed 3-fold and 4-fold coordination, and the motion of molecular "rotor". This deep learning-based approach resembles logic of a human operator, but can be scaled leading to significant shift in the way of extracting and analyzing information from raw experimental data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Structural Defects in Graphene
Banhart, F., Kotakoski, J., & Krasheninnikov, A. V. (2011) · ACS Nano · doi:10.1021/nn102598m · Cited by 3,340
Machine learning phases of matter
Carrasquilla, J., & Melko, R. G. (2017) · Nature Physics · doi:10.1038/nphys4035 · Cited by 1,448
Atom-by-atom structural and chemical analysis by annular dark-field electron microscopy
Krivanek, O. L., Chisholm, M. F., Nicolosi, V., Pennycook, T. J., Corbin, G. J., Dellby, N., Murfitt, M. F., Own, C. S., Szilagyi, Z. S., Oxley, M. P., et al. (2010) · Nature · doi:10.1038/nature08879 · Cited by 1,352
Searching for exotic particles in high-energy physics with deep learning
Baldi, P., Sadowski, P., & Whiteson, D. (2014) · Nature Communications · doi:10.1038/ncomms5308 · Cited by 1,164
Observation of polar vortices in oxide superlattices
Yadav, A. K., Nelson, C. T., Hsu, S. L., Hong, Z., Clarkson, J. D., Schlepütz, C. M., Damodaran, A. R., Shafer, P., Arenholz, E., Dedon, L. R., et al. (2016) · Nature · doi:10.1038/nature16463 · Cited by 1,024
Learning phase transitions by confusion
van Nieuwenburg, E. P. L., Liu, Y.-H., & Huber, S. D. (2017) · Nature Physics · doi:10.1038/nphys4037 · Cited by 739
Direct Imaging of Covalent Bond Structure in Single-Molecule Chemical Reactions
de Oteyza, D. G., Gorman, P., Chen, Y.-C., Wickenburg, S., Riss, A., Mowbray, D. J., Etkin, G., Pedramrazi, Z., Tsai, H.-Z., Rubio, A., et al. (2013) · Science · doi:10.1126/science.1238187 · Cited by 632
Structure and bonding at the atomic scale by scanning transmission electron microscopy
Muller, D. A. (2009) · Nature Materials · doi:10.1038/nmat2380 · Cited by 439
(1965) · Physics Today · doi:10.1063/1.3047206 · Cited by 412
Machine learning approaches in medical image analysis: From detection to diagnosis
de Bruijne, M. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.032 · Cited by 347
Machine learning in cell biology – teaching computers to recognize phenotypes
Sommer, C., & Gerlich, D. W. (2013) · Journal of Cell Science · doi:10.1242/jcs.123604 · Cited by 307
Large scale atmospheric pressure chemical vapor deposition of graphene
Vlassiouk, I., Fulvio, P., Meyer, H., Lavrik, N., Dai, S., Datskos, P., & Smirnov, S. (2013) · Carbon · doi:10.1016/j.carbon.2012.11.003 · Cited by 275
Observing Atomic Collapse Resonances in Artificial Nuclei on Graphene
Wang, Y., Wong, D., Shytov, A. V., Brar, V. W., Choi, S., Wu, Q., Tsai, H.-Z., Regan, W., Zettl, A., Kawakami, R. K., et al. (2013) · Science · doi:10.1126/science.1234320 · Cited by 254
Direct Determination of the Chemical Bonding of Individual Impurities in Graphene
Zhou, W., Kapetanakis, M. D., Prange, M. P., Pantelides, S. T., Pennycook, S. J., & Idrobo, J.-C. (2012) · Physical Review Letters · doi:10.1103/physrevlett.109.206803 · Cited by 245
Ramasse, Q. M., Seabourne, C. R., Kepaptsoglou, D.-M., Zan, R., Bangert, U., & Scott, A. J. (2013) · Nano Letters · doi:10.1021/nl304187e · Cited by 220
Scanning Transmission Electron Microscopy
Nellist, P. D. (2008) · Science of Microscopy · doi:10.1007/978-0-387-49762-4_2 · Cited by 217
Chi, M., Wang, C., Lei, Y., Wang, G., Li, D., More, K. L., Lupini, A., Allard, L. F., Markovic, N. M., & Stamenkovic, V. R. (2015) · Nature Communications · doi:10.1038/ncomms9925 · Cited by 215
Placing single atoms in graphene with a scanning transmission electron microscope
Dyck, O., Kim, S., Kalinin, S. V., & Jesse, S. (2017) · Applied Physics Letters · doi:10.1063/1.4998599 · Cited by 159
Self-citationIsoelectronic Tungsten Doping in Monolayer MoSe 2 for Carrier Type Modulation
Li, X., Lin, M., Basile, L., Hus, S. M., Puretzky, A. A., Lee, J., Kuo, Y., Chang, L., Wang, K., Idrobo, J. C., et al. (2016) · Advanced Materials · doi:10.1002/adma.201601991 · Cited by 118
Self-citationIshikawa, R., Mishra, R., Lupini, A. R., Findlay, S. D., Taniguchi, T., Pantelides, S. T., & Pennycook, S. J. (2014) · Physical Review Letters · doi:10.1103/physrevlett.113.155501 · Cited by 109
Suppression of Defects and Deep Levels Using Isoelectronic Tungsten Substitution in Monolayer MoSe2
Li, X., Puretzky, A. A., Sang, X., Santosh, K. C., Tian, M., Ceballos, F., Mahjouri‐Samani, M., Wang, K., Unocic, R. R., Zhao, H., et al. (2016) · Advanced Functional Materials · doi:10.1002/adfm.201603850 · Cited by 104
Self-citationJesse, S., He, Q., Lupini, A. R., Leonard, D. N., Oxley, M. P., Ovchinnikov, O., Unocic, R. R., Tselev, A., Fuentes‐Cabrera, M., Sumpter, B. G., et al. (2015) · Small · doi:10.1002/smll.201502048 · Cited by 97
Self-citationKalinin, S. V., Borisevich, A. Y., & Jesse, S. (2016) · Nature · doi:10.1038/539485a · Cited by 97
Self-citationOxygen-Vacancy-Induced Polar Behavior in (LaFeO3)2/(SrFeO3) Superlattices
Mishra, R., Kim, Y., Salafranca, J., Kim, S. K., Chang, S. H., Bhattacharya, A., Fong, D. D., Pennycook, S. J., Pantelides, S. T., & Borisevich, A. Y. (2014) · Nano Letters · doi:10.1021/nl500601d · Cited by 62
Functional and Spectroscopic Measurements with Scanning Tunneling Microscopy
Moore, A. M., & Weiss, P. S. (2008) · Annual Review of Analytical Chemistry · doi:10.1146/annurev.anchem.1.031207.112932 · Cited by 60
Kalinin, S. V., & Pennycook, S. J. (2017) · MRS Bulletin · doi:10.1557/mrs.2017.186 · Cited by 34
Self-citationInterplay of chemical disorder and electronic inhomogeneity in unconventional superconductors
Zeljkovic, I., & Hoffman, J. E. (2013) · Physical Chemistry Chemical Physics · doi:10.1039/c3cp51387d · Cited by 28
Identifying local structural states in atomic imaging by computer vision
Laanait, N., Ziatdinov, M. A., He, Q., et al. (2016) · Advanced Structural and Chemical Imaging · doi:10.1186/s40679-016-0028-8 · Cited by 17
Self-citationScanning Tunneling Spectroscopy
Morgenstern, M. (2010) · Lecture notes in physics · doi:10.1007/978-3-642-14376-2_5 · Cited by 6
Cited by
Machine Learning for Catalysis Informatics: Recent Applications and Prospects
Toyao, T., Maeno, Z., Takakusagi, S., Kamachi, T., Takigawa, I., & Shimizu, K.-I. (2020) · ACS Catalysis · doi:10.1021/acscatal.9b04186 · Cited by 617
Liu, Y., Han, F., Li, F., Zhao, Y., Chen, M., Xu, Z., Zheng, X., Hu, H., Yao, J., Guo, T., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-10406-7 · Cited by 551
Opportunities and Challenges for Machine Learning in Materials Science
Morgan, D., & Jacobs, R. (2020) · Annual Review of Materials Research · doi:10.1146/annurev-matsci-070218-010015 · Cited by 549
Deep learning in nano-photonics: inverse design and beyond
Wiecha, P. R., Arbouet, A., Girard, C., & Muskens, O. L. (2021) · Photonics Research · doi:10.1364/prj.415960 · Cited by 501
Deep learning for accelerated all-dielectric metasurface design
Nadell, C. C., Huang, B., Malof, J. M., & Padilla, W. J. (2019) · Optics Express · doi:10.1364/oe.27.027523 · Cited by 450
Metadata
- Authors
- Ziatdinov, M., Dyck, O., Maksov, A., Li, X., Sang, X., Xiao, K., Unocic, R. R., Vasudevan, R., Jesse, S., & Kalinin, S. V.
- Venue
- ACS Nano
- Publisher
- American Chemical Society (ACS)
- Published
- 2017-12-14
- Volume
- 11
- Issue
- 12
- Pages
- 12742-12752
- ISSN
- 1936-0851 (ISSN-L)
- 1936-086X
- OpenAlex
- W2771733300
- Cited by
- 421
- References
- 36
- FWCI
- 11.89
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.