PreprintCited by 182Machine Learning in Materials ScienceAdvanced Chemical Physics StudiesComputational Drug Discovery Methods

DPA-2: a large atomic model as a multi-task learner

Open access
Zhang, Duo, Liu, Xinzijian, Zhang, Xiangyu, Zhang, Chengqian, Cai, Chun, Bi, Hangrui, Du, Yiming, Qin, Xuejian +35 more·2023·arXiv

Abstract

The rapid advancements in artificial intelligence (AI) are catalyzing transformative changes in atomic modeling, simulation, and design. AI-driven potential energy models have demonstrated the capability to conduct large-scale, long-duration simulations with the accuracy of ab initio electronic structure methods. However, the model generation process remains a bottleneck for large-scale applications. We propose a shift towards a model-centric ecosystem, wherein a large atomic model (LAM), pre-trained across multiple disciplines, can be efficiently fine-tuned and distilled for various downstream tasks, thereby establishing a new framework for molecular modeling. In this study, we introduce the DPA-2 architecture as a prototype for LAMs. Pre-trained on a diverse array of chemical and materials systems using a multi-task approach, DPA-2 demonstrates superior generalization capabilities across multiple downstream tasks compared to the traditional single-task pre-training and fine-tuning methodologies. Our approach sets the stage for the development and broad application of LAMs in molecular and materials simulation research.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2312.15492

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Universal MLPs are commonly trained on datasets comprising only close-to-equilibrium structures, so caution is needed when applying them to activated events.

    “Also, commonly, the universal MLPs are trained on datasets comprising only close-to-equilibrium structures25,27–29, hence caution is needed when applying them to activated events.”
  • Universal MLPs cover large parts of the chemical space with dozens of elements but have limited coverage of the configuration space, e.g., not considering all relevant chemical reactions with associated transition states.

    “Two paradigms dominate the MLP field currently: (i) training of universal MLP that cover large parts of the chemical space with dozens of elements, but with limited coverage of the configuration space, e.g., not considering all relevant chemical reactions with the associated transition states, e.g., OC22, CHGNet, and others25–29”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Zhang, D., Liu, X., Zhang, X., Zhang, C., Cai, C., Bi, H., Du, Y., Qin, X., Peng, A., Huang, J., Li, B., Shan, Y., Zeng, J., Zhang, Y., Liu, S., Li, Y., Chang, J., Wang, X., Zhou, S., Liu, J., Luo, X., Wang, Z., Jiang, W., Wu, J., Yang, Y., Yang, J., Yang, M., Gong, F.-Q., Zhang, L., Shi, M., Dai, F.-Z., York, D. M., Liu, S., Zhu, T., Zhong, Z., Lv, J., Cheng, J., Jia, W., Chen, M., Ke, G., E, W., Zhang, L., & Wang, H.
Affiliations
  • Peking University (CN)
  • Chinese Academy of Sciences (CN)
  • East China Normal University (CN)
  • Rutgers, The State University of New Jersey (US)
  • Westlake University (CN)
  • Princeton University (US)
  • Xihua University (CN)
  • Jilin University (CN)
  • Italian Institute of Technology (IT)
  • Xiamen University (CN)
  • New York University Shanghai (CN)
Corresponding author
  • Zhang, Linshuang
  • Zhang, Linfeng
  • Wang, Han
Venue
arXiv
Publisher
arXiv
Published
2023
Volume
10
Issue
1
ISSN
2057-3960 (ISSN-L)
OpenAlex
W4405574099
Cited by
155
References
81
Citations by year
2026: 63 · 2025: 86 · 2024: 6
FWCI
15.10
Type
journal article
Language
en
Open access
Locations
  • npj Computational Materials (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (2209718)
  • U.S. Department of Energy (DESC0019394)
  • National Natural Science Foundation of China (92270001, 91961204, 22225302, 12122103, 12288101, 12122401, 12074007, 22173032, 12034009, 22222303, 12135002)
  • National Institutes of Health
  • National Science Fund for Distinguished Young Scholars (22225302)
  • National Institute of General Medical Sciences (R01GM107485)
  • Natural Science Foundation of Zhejiang Province (2022XHSJJ006)
Topics
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Advanced Chemical Physics Studies
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
Keywords
multi-task learning, pre-training, fine-tuning, knowledge distillation, molecular simulation, materials simulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review