The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
A multi-modal pre-training transformer for universal transfer learning in metal–organic frameworks
Open accessAbstract
Metal–organic frameworks (MOFs) are a class of crystalline porous materials that exhibit a vast chemical space owing to their tunable molecular building blocks with diverse topologies. An unlimited number of MOFs can, in principle, be synthesized. Machine learning approaches can help to explore this vast chemical space by identifying optimal candidates with desired properties from structure–property relationships. Here we introduce MOFTransformer, a multi-modal Transformer encoder pre-trained with 1 million hypothetical MOFs. This multi-modal model utilizes integrated atom-based graph and energy-grid embeddings to capture both local and global features of MOFs, respectively. By fine-tuning the pre-trained model with small datasets ranging from 5,000 to 20,000 MOFs, our model achieves state-of-the-art results for predicting across various properties including gas adsorption, diffusion, electronic properties, and even text-mined data. Beyond its universal transfer learning capabilities, MOFTransformer generates chemical insights by analyzing feature importance through attention scores within the self-attention layers. As such, this model can serve as a platform for other MOF researchers that seek to develop new machine learning models for their work. Metal–organic frameworks are of high interest for a range of energy and environmental applications due to their stable gas storage properties. A new machine learning approach based on a pre-trained multi-modal transformer can be fine-tuned with small datasets to predict structure-property relationships and design new metal-organic frameworks for a range of specific tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Fast Parallel Algorithms for Short-Range Molecular Dynamics
Plimpton, S. (1995) · Journal of Computational Physics · doi:10.1006/jcph.1995.1039 · Cited by 44,721
Metal-organic framework materials as catalysts
Lee, J., Farha, O. K., Roberts, J., Scheidt, K. A., Nguyen, S. T., & Hupp, J. T. (2009) · Chemical Society Reviews · doi:10.1039/b807080f · Cited by 7,969
Xie, T., & Grossman, J. C. (2018) · Physical Review Letters · doi:10.1103/physrevlett.120.145301 · Cited by 2,828
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
Kim, W., Son, B., & Kim, I. (2021) · arXiv · doi:10.48550/arxiv.2102.03334 · Cited by 951
Data-driven design of metal–organic frameworks for wet flue gas CO2 capture
Boyd, P. G., Chidambaram, A., García-Díez, E., Ireland, C. P., Daff, T. D., Bounds, R., Gładysiak, A., Schouwink, P., Moosavi, S. M., Maroto-Valer, M. M., et al. (2019) · Nature · doi:10.1038/s41586-019-1798-7 · Cited by 885
Self-citationUnified Vision-Language Pre-Training for Image Captioning and VQA
Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7005 · Cited by 864
Chung, Y. G., Haldoupis, E., Bucior, B. J., Haranczyk, M., Lee, S., Zhang, H., Vogiatzis, K. D., Milisavljevic, M., Ling, S., Camp, J. S., et al. (2019) · Journal of Chemical & Engineering Data · doi:10.1021/acs.jced.9b00835 · Cited by 863
Cited by
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
Han, Z., Yang, Y., Rushlow, J., Huo, J., Liu, Z., Hsu, Y.-C., Yin, R., Wang, M., Liang, R., Wang, K.-Y., et al. (2025) · Chemical Society Reviews · doi:10.1039/d4cs00432a · Cited by 272
Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review
Yang, M., Shi, N., Chen, H., Po, E. Y. P., Ng, P. Y., Lai, P. C. K., Wu, Y.-C., & Fong, D. Y. T. (2026) · Health Data Science · doi:10.34133/hds.0356 · Cited by 3
“Furthermore, techniques such as transfer learning [ 116 , 117 ] and federated learning [ 118 , 119 ] are encouraged to improve the robustness and generalizability of multimodal models, with federated learning also protecting data privacy.”
Metadata
- Authors
- Kang, Y., Park, H., Smit, B., & Kim, J.
- Venue
- Nature Machine Intelligence
- Published
- 2023-03-13
- Volume
- 5
- Issue
- 3
- Pages
- 309-318
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W4324032752
- Cited by
- 208
- References
- 13
- FWCI
- 14.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A multi-modal pre-training transformer for universal transfer learning in metal–organic frameworks. https://app.cochat.ai/papers/10.1038/s42256-023-00628-2
- The paper
- Kang, Y., Park, H., Smit, B., & Kim, J. (2023). A multi-modal pre-training transformer for universal transfer learning in metal–organic frameworks. Nature Machine Intelligence. https://doi.org/10.1038/s42256-023-00628-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.