CoChat
Sign Up
PreprintCited by 5Machine Learning in Materials ScienceTopic ModelingMultimodal Machine Learning Applications

Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation

Open access
Yan, Keqiang, Li, Xiner, Ling, Hongyi, Ashen, Kenna, Edwards, Carl, Arróyave, Raymundo, Zitnik, Marinka, Ji, Heng +3 more·2025·arXiv
DOI 10.48550/arxiv.2503.00152arXiv 2503.00152
Open this paper in CoChat — free accountOpen PDF

Abstract

We consider the problem of crystal materials generation using language models (LMs). A key step is to convert 3D crystal structures into 1D sequences to be processed by LMs. Prior studies used the crystallographic information framework (CIF) file stream, which fails to ensure SE(3) and periodic invariance and may not lead to unique sequence representations for a given crystal structure. Here, we propose a novel method, known as Mat2Seq, to tackle this challenge. Mat2Seq converts 3D crystal structures into 1D sequences and ensures that different mathematical descriptions of the same crystal are represented in a single unique sequence, thereby provably achieving SE(3) and periodic invariance. Experimental results show that, with language models, Mat2Seq achieves promising performance in crystal structure generation as compared with prior methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.00152

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent works exemplify advanced tokenization methods in the biomedical domain, including protein, genomic, and chemical structure tokenizers.

    “Recent works exemplify the transformative potential of advanced tokenization methods in the biomedical domain, including protein [38, 128, 170], genomic [30], and chemical structure [160] tokenizers.”

    From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [160]

    Section 3 (p. 3) describes Mat2Seq as a tokenizer that converts 3D crystal structures into sequences, supporting its use as an example of a structure tokenizer. The paper studies crystalline materials, however, and does not establish the claimed biomedical application. The protein and genomic examples have separate citations in the citing sentence and are not attributed to this source. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 37

  1. [1][1] Ioannis Petousis, Wei Chen, Geoffroy Hautier, Tanja Graf, Thomas D Schladt, Kristin A Persson, and Fritz B Prinz. Benchmarking density functional perturbation theory to enable high-throughput screening of materials for dielectric constant and refractive index. Physical Review B , 93(11):115151, 2016. -
  2. [2][2] Susmriti Das Mahapatra, Preetam Chandan Mohapatra, Adrianus Indrat Aria, Graham Christie, Yogendra Kumar Mishra, Stephan Hofmann, and Vijay Kumar Thakur. Piezoelectric Materials for Energy Harvesting and Sensing Applications: Roadmap for Future Smart Materials. Advanced Science , 8(17):2100864, 2021. -
  3. [3][3] Hua Wang and Xiaofeng Qian. Ferroelectric nonlinear anomalous Hall effect in few-layer WTe 2 . npj Computational Materials , 5(1):119, 2019. -
  4. [4][4] Jun Xiao, Ying Wang, Hua Wang, C. D. Pemmaraju, Siqi Wang, Philipp Muscher, Edbert J. Sie, Clara M. Nyby, Thomas P. Devereaux, Xiaofeng Qian, Xiang Zhang, and Aaron M. Lindenberg. Berry curvature memory through electrically driven stacking transitions. Nature Physics , 16(10):1028-1034, 2020. -
  5. [6][6] Kamal Choudhary, Daniel Wines, Kangming Li, Kevin F Garrity, Vishu Gupta, Aldo H Romero, Jaron T Krogel, Kayahan Saritas, Addis Fuhr, Panchapakesan Ganesh, et al. JARVISLeaderboard: a large scale benchmark of materials design methods. npj Computational Materials , 10(1):93, 2024. -
  6. [7][7] Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, Tianfan Fu, Yucheng Wang, Haiyang Yu, YuQing Xie, Xiang Fu, Alex Strasser, Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hannah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards, Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang, Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang, Tuong Phung, Minkai Xu, Chaitanya K. Joshi, Simon V. Mathis, Kamyar Azizzadenesheli, Ada Fang, Alán Aspuru-Guzik, Erik Bekkers, Michael Bronstein, Marinka Zitnik, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovec, Heng Ji, Jimeng Sun, Regina Barzilay, Tommi Jaakkola, Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess Smidt, and Shuiwang Ji. Artificial intelligence for science in quantum, atomistic, and continuum systems. arXiv preprint arXiv:2307.08423 , 2023. - · arXiv
  7. [8][8] Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal material property prediction. In The 36th Annual Conference on Neural Information Processing Systems , pages 15066-15080, 2022. -
  8. [9][9] Yuchao Lin, Keqiang Yan, Youzhi Luo, Yi Liu, Xiaoning Qian, and Shuiwang Ji. Efficient approximations of complete interatomic potentials for crystal property prediction. In ICML , pages 21260-21287, 2023. -
  9. [10][10] Daniel Flam-Shepherd and Alán Aspuru-Guzik. Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB files. arXiv preprint arXiv:2305.05708 , 2023. - · arXiv
  10. [11][11] Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C Lawrence Zitnick, and Zachary Ulissi. Fine-tuned language models generate stable inorganic materials as text. arXiv preprint arXiv:2402.04379 , 2024. - · arXiv
  11. [12][12] Luis M Antunes, Keith T Butler, and Ricardo Grau-Crespo. Crystal structure generation with autoregressive large language modeling. arXiv preprint arXiv:2307.04340 , 2023. - · arXiv
  12. [13][13] Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3D molecular graphs. In The 36th Annual Conference on Neural Information Processing Systems , pages 650-664, 2022. -
  13. [14][14] Keqiang Yan, Cong Fu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Complete and efficient graph transformers for crystal material property prediction. In The Twelfth International Conference on Learning Representations , 2024. -
  14. [15][15] Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal Diffusion Variational Autoencoder for Periodic Material Generation. In International Conference on Learning Representations , 2022. -
  15. [16][16] Youzhi Luo, Chengkai Liu, and Shuiwang Ji. Towards symmetry-aware generation of periodic materials. Advances in Neural Information Processing Systems , 36, 2024. -
  16. [17][17] Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal structure prediction by joint equivariant diffusion. Advances in Neural Information Processing Systems , 36, 2024. -
  17. [18][18] Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al. MatterGen: a generative model for inorganic materials design. arXiv preprint arXiv:2312.03687 , 2023. - · arXiv
  18. [19][19] Hong-Long Shi and Zi-An Li. Niggli reduction and bravais lattice determination. Journal of Applied Crystallography , 55(1):204-210, 2022. -
  19. [21][21] Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833 , 2018. - · arXiv
  20. [22][22] David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. A learning algorithm for boltzmann machines. Cognitive Science , 9(1):147-169, 1985. -
  21. [23][23] Ivano E Castelli, David D Landis, Kristian S Thygesen, Søren Dahl, Ib Chorkendorff, Thomas F Jaramillo, and Karsten W Jacobsen. New cubic perovskites for one-and two-photon water splitting using the computational materials repository. Energy & Environmental Science , 5(10):9034-9043, 2012. -
  22. [24][24] Ivano E Castelli, Thomas Olsen, Soumendu Datta, David D Landis, Søren Dahl, Kristian S Thygesen, and Karsten W Jacobsen. Computational screening of perovskite metal oxides for optimal solar light capture. Energy & Environmental Science , 5(2):5814-5819, 2012. -
  23. [25][25] Chris J Pickard, Ashkan Salamat, Michael J Bojdys, Richard J Needs, and Paul F McMillan. Carbon nitride frameworks and dense crystalline polymorphs. Physical Review B , 94(9):094104, 2016. -
  24. [26][26] Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials , 1(1):011002, 2013. -
  25. [28][28] Benjamin Kurt Miller, Ricky TQ Chen, Anuroop Sriram, and Brandon M Wood. FlowMM: Generating Materials with Riemannian Flow Matching. In Forty-first International Conference on Machine Learning , 2024. -
  26. [29][29] Dejan Zagorac, H Müller, S Ruehl, J Zagorac, and Silke Rehme. Recent developments in the Inorganic Crystal Structure Database: theoretical crystal structure data and related features. Journal of Applied Crystallography , 52(5):918-925, 2019. -
  27. [31][31] Markus Scheidgen, Lauri Himanen, Alvin Noe Ladines, David Sikter, Mohammad Nakhaee, Ádám Fekete, Theodore Chang, Amir Golparvar, José A Márquez, Sandor Brockhauser, et al. NOMAD: A distributed web-based platform for managing materials science research data. Journal of Open Source Software , 8(90):5388, 2023. -
  28. [32][32] Alexandra Plumhoff. Thermodynamic properties, crystal structures, phase relations and isotopic studies of selected copper oxysalts . PhD thesis, 2020. -
  29. [33][33] Elizabeth A Pogue, Jack Bond, Cassandra Imperato, John BS Abraham, Natalia Drichko, and Tyrel M McQueen. A Gold(I) Oxide Double Perovskite: Ba 2 AuIO 6 . Journal of the American Chemical Society , 143(45):19033-19042, 2021. -
  30. [34][34] Matthias Weil. Ca 2 Te 3 O 8 , a new phase in the CaO-TeO 2 system. Acta Crystallographica Section E: Crystallographic Communications , 75(1):26-29, 2019. -
  31. [35][35] Griffen Desroches and Svilen Bobev. Synthesis and structure determination of Ce 6 Cd 23 Te: a new chalcogen-containing member of the RE 6 Cd 23 T family ( RE is a rare-earth metal and T is a late group 14, 15 and 16 element). Acta Crystallographica Section C: Structural Chemistry , 73(2):121-125, 2017. -
  32. [36][36] Gregory Morrison, Virginia G Jones, K Pilar Zamorano, John E Greedan, and Hans-Conrad Zur Loye. Flux Synthesis, UV-vis Absorbance, and Magnetism of Cesium Copper Silicates with an Isolated Super-Super Exchange Spin Dimer in Cs 6 Cu 2 Si 9 O 23 . Inorganic Chemistry , 62(29):11682-11689, 2023. -
  33. [37][37] Andrew J Craig, Stanislav S Stoyko, Allyson Bonnoni, and Jennifer A Aitken. Syntheses and crystal structures of the quaternary thiogermanates Cu 4 FeGe 2 S 7 and Cu 4 CoGe 2 S 7 . Acta Crystallographica Section E: Crystallographic Communications , 76(7):1117-1121, 2020. -
  34. [38][38] Nan Zhang, Xiao Huang, Wen-Dong Yao, Yao Chen, Zheng-Rui Pan, Bingxuan Li, Wenlong Liu, and Sheng-Ping Guo. Eu 2 MGe 2 OS 6 (M= Mn, Fe, Co): Three Melilite-Type Rare-Earth Oxythiogermanates Exhibiting Balanced Nonlinear-Optical Behaviors. Inorganic Chemistry , 62(40):16299-16303, 2023. -
  35. [39][39] Weimin Dong, Yingjie Sun, Henghao Feng, Dazheng Deng, Jun Jiang, Jin Yang, Wei Guo, Libin Tang, Jincheng Kong, and Jun Zhao. K 2 Sr 4 (PO 3 ) 10 : A Polyphosphate with Deep-UV Cutoff Edge and Enlarged Birefringence. Inorganic Chemistry , 62(39):16215-16221, 2023. -
  36. [40][40] Hong Yan, Kotaro Fujii, Houria Kabbour, Akira Chikamatsu, Yu Meng, Yoshitaka Matsushita, Masatomo Yashima, Kazunari Yamaura, and Yoshihiro Tsujimoto. La 4 Ga 2 S 83 : A Rare-Earth Gallium Oxysulfide with Disulfide Ions. Inorganic Chemistry , 62(26):10481-10489, 2023. -
  37. [41][41] Volodymyr Pavlyuk, Grygoriy Dmytriv, Ivan Tarasiuk, and Helmut Ehrenberg. Li 9 Al 4 Sn 5 as a new ordered superstructure of the Li 13 Sn 5 type. Acta Crystallographica Section C: Structural Chemistry , 73(4):337-342, 2017. -

Cited by

Sorted by citations

Authors
Yan, K., Li, X., Ling, H., Ashen, K., Edwards, C., Arróyave, R., Zitnik, M., Ji, H., Qian, X., Qian, X., & Ji, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
125050-125072
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2503.00152
arXiv
2503.00152
OpenAlex
W4415795360
Cited by
5
References
0
Citations by year
2026: 3 · 2025: 2
FWCI
9.23
Type
conference paper
Language
en
Open access
Licence
creativecommons.org/licenses/by-nc-sa/4.0/legalcode
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
tokenization, crystalline materials, language models, crystal structure representation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review