RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Multi-step retrosynthetic route planning (MRRP) is the core task in synthetic chemistry, in which chemists recursively deconstruct a target molecule to find a set of reactants that make up the target. MRRP is challenging in that the search space is vast, and chemists are often lost in the process. Existing AI models can achieve automatic MRRP fast, but they only work on relatively simple targets, which leaves complex molecules under chemists’ expertise. To facilitate MRRP of complex molecules, we proposed a human-AI collaborative system, RetroLens, through a participatory design process. AI can contribute by two approaches: joint action and algorithm-in-the-loop. Deconstruction steps are allocated to chemists or AI based on their capabilities and AI recommends candidate revision steps to fix problems along the way. A within-subjects study (N=18) showed that chemists who used RetroLens reported faster MRRP, broader design space exploration, higher confidence in their planning, and lower cognitive load.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, AI's real-world applications are not infallible, still far from 100% accuracy [44, 86, 109].”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [109]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
ZINC 15 – Ligand Discovery for Everyone
Sterling, T., & Irwin, J. J. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00559 · Cited by 3,224
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 4×
“Prior work proposed human-AI interaction design guidelines suggesting that AI systems should help human “make clear how well the system can do what it can do” when users initially interact with AI [2].”
BackgroundMethodInfluentialExplaining machine learning classifiers through diverse counterfactual explanations
Mothilal, R. K., Sharma, A., & Tan, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372850 · Cited by 1,233
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“, the user could design a better retrosynthetic route for a product on their own than AI but accept AI’s recommendation [12]) [43, 70].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Computer‐Assisted Synthetic Planning: The End of the Beginning
Szymkuć, S., Gajewska, E. P., Klucznik, T., Molga, K., Dittwald, P., Startek, M., Bajczyk, M., & Grzybowski, B. A. (2016) · Angewandte Chemie International Edition · doi:10.1002/anie.201506101 · Cited by 611
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · cited 1×
“be delegated to AI only upon users’ request even if the molecules are within the scope of AI’s capability [27].”
BackgroundRetrosynthetic Reaction Prediction Using Neural Sequence-to-Sequence Models
Liu, B., Ramsundar, B., Kawthekar, P., Shi, J., Gomes, J., Luu Nguyen, Q., Ho, S., Sloane, J., Wender, P., & Pande, V. (2017) · ACS Central Science · doi:10.1021/acscentsci.7b00303 · Cited by 536
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271 · cited 1×
“developed AI-steering tools enabling novices to direct the music generation model to iteratively create music [58].”
BackgroundHuman-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Lai, V., Carton, S., Bhatnagar, R., Liao, Q. V., Zhang, Y., & Tan, C. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501999 · Cited by 145
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123
Chematica: A Story of Computer Code That Started to Think like a Chemist
Grzybowski, B. A., Szymkuć, S., Gajewska, E. P., Molga, K., Dittwald, P., Wołos, A., & Klucznik, T. (2018) · Chem · doi:10.1016/j.chempr.2018.02.024 · Cited by 114 · cited 1×
“It which reactions they choose may have caused the route failure, usually takes weeks to plan a retrosynthetic route for a single mol- they still need to evaluate the trade-ofs among a set of candidate ecule [32].”
BackgroundExploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers
Tsai, C.-H., You, Y., Gui, X., Kou, Y., & Carroll, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445101 · Cited by 86 · cited 1×
“provide the reaction precedents in training datasets similar to the routes [16, 18, 36, 48] and the data sources of these precedents [45, 89], making it easier for users to understand and evaluate these routes.”
BackgroundA Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy
Park, J. S., Barber, R., Kirlik, A., & Karahalios, K. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359204 · Cited by 65
Ma, S., Wei, Z., Tian, F., Fan, X., Zhang, J., Shen, X., Lin, Z., Huang, J., Měch, R., Samaras, D., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300701 · Cited by 24
Self-citationToward Scalable and Unified Example-Based Explanation and Outlier Detection
Chong, P., Cheung, N.-M., Elovici, Y., & Binder, A. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3127847 · Cited by 14
Research works printed but not linked: 79
- Laura KG Ackerman-Biegasiewicz, Daniela M Arias-Rotondo, Kyle F Biegasiewicz, Elizabeth Elacqua, Matthew R Golder, Laure V Kayser, Jessica R Lamb, Christine M Le, Nathan A Romero, Sidney M Wilkerson-Hill, 2020. Organic Chemistry: A Retrosynthetic Approach to a Diverse Field., 1845–1850 pages.
- 10.1145/3415167 · DOI
- 10.1038/s41573-020-00114-z · DOI · Paper page
- 10.1039/C8SC05611K · DOI
- 10.1021/acs.jcim.8b00801 · DOI
- 10.1038/s41598-021-81889-y · DOI
- 10.1145/3377325.3377497 · DOI
- 10.1080/2159676X.2019.1628806 · DOI · Paper page
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- Liwei Chan, Yi-Chi Liao, George B Mo, John J Dudley, Chun-Lien Cheng, Per Ola Kristensson, and Antti Oulasvirta. 2022. Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques. In CHI Conference on Human Factors in Computing Systems. 1–14.
- Binghong Chen, Chengtao Li, Hanjun Dai, and Le Song. 2020. Retro*: learning retrosynthetic planning with neural guided A* search. In International Conference on Machine Learning. PMLR, 1608–1616.
- Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. 2019. This looks like that: deep learning for interpretable image recognition. Advances in Neural Information Processing Systems 32 (2019).
- 10.1145/2556288.2557375 · DOI
- 10.1021/acs.jcim.7b00622 · DOI · Paper page
- Elias James Corey. 1967. General methods for the construction of complex molecules. Pure and Applied chemistry 14, 1 (1967), 19–38.
- Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. 2019. Retrosynthesis prediction with conditional graph logic network. Advances in Neural Information Processing Systems 32 (2019).
- Ian W Davies. 2019. The digitization of organic synthesis. Nature 570, 7760 (2019), 175–181.
- 10.1145/3185517 · DOI · Paper page
- William Finnigan, Sabine L Flitsch, Lorna J Hepworth, and Nicholas J Turner. 2021. Enzyme Cascade Design: Retrosynthesis Approach. In Enzyme Cascade Design and Modelling. Springer, 7–30.
- 10.1186/s13321-020-00472-1 · DOI · Paper page
- 10.1016/j.tetlet.2008.03.107 · doi:10.1016/j.tetlet.2008.03.107
- 10.1109/TVCG.2013.173 · DOI · Paper page
- 10.1016/j.joep.2016.12.001 · DOI
- 10.1609/aaai.v36i4.20318 · DOI
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in Psychology. Vol. 52. Elsevier, 139–183.
- 10.1609/hcomp.v7i1.5265 · DOI
- 10.1021/acs.jcim.1c01074 · DOI · Paper page
- Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. 1–19.
- Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. 2018. To trust or not to trust a classifier. Advances in Neural Information Processing Systems 31 (2018).
- Ekaterina Jussupow Izak Benbasat and Armin Heinzl. 2020. Why are we averse towards algorithms? A comprehensive literature review on algorithm aversion. (2020).
- 10.1145/2856767.2856789 · DOI
- 10.1007/978-3-030-30493-5_78 · DOI
- Harmanpreet Kaur, Alex Williams, and WS Lasecki. 2019. Building shared mental models between humans and ai for effective collaboration. CHI’19, May 2019, Glasgow, Scotland(2019).
- 10.1145/3491102.3502116 · DOI · Paper page
- 10.1016/j.chempr.2018.02.002 · DOI · Paper page
- 10.1007/11871842_29 · DOI · Paper page
- Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International Conference on Machine Learning. PMLR, 1885–1894.
- 10.1145/3290605.3300742 · DOI
- 10.1145/3411764.3445474 · DOI
- 10.1039/C9SC03666K · DOI · Paper page
- 10.1145/2207676.2208693 · doi:10.1145/2207676.2208693
- Shuai Ma, Mingfei Sun, and Xiaojuan Ma. 2022. Modeling Adaptive Expression of Robot Learning Engagement and Exploring its Effects on Human Teachers. ACM Transactions on Computer-Human Interaction (2022).
- 10.1145/3491102.3517482 · DOI
- 10.1145/3359297 · DOI · Paper page
- Barbara Mikulak-Klucznik, Patrycja Gołębiowska, Alison A Bayly, Oskar Popik, Tomasz Klucznik, Sara Szymkuć, Ewa P Gajewska, Piotr Dittwald, Olga Staszewska-Krajewska, Wiktor Beker, 2020. Computational planning of the synthesis of complex natural products. Nature 588, 7836 (2020), 83–88.
- 10.1145/3411764.3445096 · DOI · Paper page
- 10.2533/chimia.2019.997 · DOI
- 10.1109/TVCG.2016.2598589 · DOI
- 10.1021/ci980147y · DOI
- 10.1021/acscentsci.9b00055 · DOI · Paper page
- 10.1039/C9SC05704H · DOI · Paper page
- 10.1093/bioinformatics/btaa219 · DOI
- Marwin HS Segler, Mike Preuss, and Mark P Waller. 2018. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 7698 (2018), 604–610.
- Marwin HS Segler and Mark P Waller. 2017. Modelling chemical reasoning to predict and invent reactions. Chemistry–A European Journal 23, 25 (2017), 6118–6128.
- 10.1609/aaai.v35i1.16131 · DOI
- 10.1145/3491102.3517659 · DOI
- Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. 2020. A graph to graphs framework for retrosynthesis prediction. In International Conference on Machine Learning. PMLR, 8818–8827.
- Sarah Jean Stevens. 2011. Progress toward the synthesis of providencin. Ph. D. Dissertation. Colorado State University.
- Dong Sun, Renfei Huang, Yuanzhe Chen, Yong Wang, Jia Zeng, Mingxuan Yuan, Ting-Chuen Pong, and Huamin Qu. 2019. PlanningVis: A visual analytics approach to production planning in smart factories. IEEE Transactions on Visualization and Computer Graphics 26, 1(2019), 579–589.
- 10.1016/j.eswa.2009.05.005 · DOI · Paper page
- 10.1145/3306618.3314293 · DOI · Paper page
- 10.1038/s41467-020-19266-y · DOI · Paper page
- 10.1145/3097983.3098039 · DOI
- 10.1007/s40593-015-0056-x · DOI
- 10.1016/j.molliq.2017.12.055 · doi:10.1016/j.molliq.2017.12.055
- 10.1109/TVCG.2017.2745078 · DOI
- 10.1145/3290605.3300831 · DOI · Paper page
- 10.1145/3025453.3025819 · doi:10.1145/3025453.3025819
- 10.1002/cjoc.202100273 · DOI
- 10.1021/ci00057a005 · DOI · Paper page
- 10.1109/TVCG.2018.2865126 · DOI
- Di Weng, Chengbo Zheng, Zikun Deng, Mingze Ma, Jie Bao, Yu Zheng, Mingliang Xu, and Yingcai Wu. 2020. Towards better bus networks: A visual analytics approach. IEEE transactions on visualization and computer graphics 27, 2(2020), 817–827.
- 10.1145/3173574.3173821 · DOI
- Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, Jinyu Yang, Yang Yu, and Junzhou Huang. 2020. Retroxpert: Decompose retrosynthesis prediction like a chemist. Advances in Neural Information Processing Systems 33 (2020), 11248–11258.
- 10.1145/3411764.3445654 · DOI
- 10.1016/S0377-2217(97)00147-1 · DOI
- 10.1145/2207676.2207708 · DOI · Paper page
- 10.1145/3025171.3025215 · doi:10.1145/3025171.3025215
- 10.1145/3491102.3517449 · DOI
Cited by
Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Hadi Mogavi, R., Peng, Z., Ma, S., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642807 · Cited by 70
Self-citationCharting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Mogavi, R. H., Peng, Z., Ma, S., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2401.14915 · Cited by 70
Self-citationA Taxonomy for Human-LLM Interaction Modes: An Initial Exploration
Gao, J., Gebreegziabher, S. A., Choo, K. T. W., Li, T. J.-J., Perrault, S. T., & Malone, T. W. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650786 · Cited by 57
“[58], the LLM can decide whether to assign tasks to each member of a group.”
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2403.09552 · Cited by 56
Self-citationMa, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
Self-citation
Metadata
- Authors
- Shi, C., Hu, Y., Wang, S., Ma, S., Zheng, C., Ma, X., & Luo, Q.
- Affiliations
- Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, China
- School of Electronic Engineering and Computer Science, Queen Mary, University of London, United Kingdom
- The Hong Kong University of Science and Technology, China
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-20
- Cited by
- 19
- References
- 28
- Funders
- The Research Grants Council of the Hong Kong Special Administrative Region under General Research Fund (16203421)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.