SciDaSynth: Interactive Structured Data Extraction From Scientific Literature With Large Language Model
Open accessAbstract
The explosion of scientific literature has made the efficient and accurate extraction of structured data a critical component for advancing scientific knowledge and supporting evidence‐based decision‐making. However, existing tools often struggle to extract and structure multimodal, varied, and inconsistent information across documents into standardized formats. We introduce SciDaSynth, a novel interactive system powered by large language models that automatically generates structured data tables according to users' queries by integrating information from diverse sources, including text, tables, and figures. Furthermore, SciDaSynth supports efficient table data validation and refinement, featuring multi‐faceted visual summaries and semantic grouping capabilities to resolve cross‐document data inconsistencies. A within‐subjects study with nutrition and NLP researchers demonstrates SciDaSynth's effectiveness in producing high‐quality structured data more efficiently than baseline methods. We discuss design implications for human–AI collaborative systems supporting data extraction tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs can identify relevant literature given mechanistic context, read scientific text, and extract information into structured templates.
“LLMs can identify relevant literature given mechanistic context, read scientific text, and extract information into structured templates [14, 15, 16, 17, 18].”
From Structured Schemas for Provenance‐Rich, <scp>LLM</scp> ‐Assisted <scp>QSP</scp> Model Calibration (2026) · cites this paper as [18]
The System > Data Extraction and Structuring section describes retrieving relevant content from a collection of scientific papers, then using LLMs to extract information into structured data tables. It does not describe identifying relevant literature from mechanistic context; its retrieval starts with a user question and searches content in papers already provided. (LLM, conf=0.86)
Experimental — accuracy not yet measuredLLMs can identify relevant literature given mechanistic context, read scientific text, and extract information into structured templates.
“LLMs can identify relevant literature given mechanistic context, read scientific text, and extract information into structured templates14,15,16,17,18.”
From Structured Schemas for Provenance-Rich, LLM-Assisted QSP Model Calibration (2026) · cites this paper as [18]
MAPLE addresses a gap between general-purpose biomedical extraction frameworks17,18, which do not address QSP-specific downstream requirements (uncertainty quantification, forward model specification, code generation), and QSP-focused tools like QSP-Copilot16, which include literature extraction components but emphasize model structure discovery rather than quantitative parameter calibration with
“MAPLE addresses a gap between general-purpose biomedical extraction frameworks17,18, which do not address QSP-specific downstream requirements (uncertainty quantification, forward model specification, code generation), and QSP-focused tools like QSP-Copilot16”
From Structured Schemas for Provenance-Rich, LLM-Assisted QSP Model Calibration (2026) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 2× · see citing works
“Unlike standard prompting, which relies solely on a model's pretrained knowledge (and can be outdated due to the LLM's training cutoff), RAG dynamically retrieves and integrates up ‐ to ‐ date, domain ‐ specific information into prompts.”
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218 · see citing works
Wu, J.-H., & Wang, S.-C. (2005) · Information & Management · doi:10.1016/j.im.2004.07.001 · Cited by 2,284 · see citing works
Structured information extraction from scientific text with large language models
Dagdelen, J., Dunn, A., Lee, S., Walker, N., Rosen, A. S., Ceder, G., Persson, K. A., & Jain, A. (2024) · Nature Communications · doi:10.1038/s41467-024-45563-x · Cited by 771 · see citing works
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685 · see citing works
RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials
Marshall, I. J., Kuiper, J., & Wallace, B. C. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv044 · Cited by 317 · see citing works
Evaluation of Retrieval-Augmented Generation: A Survey
Yu, H., Gan, A., Zhang, K., Tong, S., Liu, Q., & Liu, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-1024-2_8 · Cited by 268 · see citing works
CERMINE: automatic extraction of structured metadata from scientific literature
Tkaczyk, D., Szostek, P., Fedoryszak, M., Dendek, P. J., & Bolikowski, Ł. (2015) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-015-0249-8 · Cited by 199 · see citing works
Lewis, P., Ott, M., Du, J., & Stoyanov, V. (2020) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2020.clinicalnlp-1.17 · Cited by 175 · see citing works
RARR: Researching and revising what language models say, using language models
Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V., Lao, N., Lee, H., Juan, D.-C., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.910 · Cited by 141 · cited 2× · see citing works
“Then, automated algorithms or human raters can examine whether the reference source really supports the generated answers using different criteria (Gao et al. 2023; Yue et al. 2023; Rashkin et al. 2023; Bohnet et al. 2023; Menick et al. 2022).”
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Dasigi, P., Lo, K., Beltagy, I., Cohan, A., Smith, N. A., & Gardner, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.365 · Cited by 138 · see citing works
Clark, C., & Divvala, S. (2016) · doi:10.1145/2910896.2910904 · Cited by 118 · see citing works
Summarising good practice guidelines for data extraction for systematic reviews and meta-analysis
Taylor, K., Mahtani, K. R., & Aronson, J. K. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111651 · Cited by 116 · see citing works
Peer review of GPT-4 technical report and systems card
Gallifant, J., Fiske, A., Strekalova, Y. A., Osorio-Valencia, J. S., Parke, R. L., Mwavu, R., Martinez‐Martin, N., Gichoya, J. W., Ghassemi, M., Demner‐Fushman, D., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000417 · Cited by 115 · see citing works
Inferring Which Medical Treatments Work from Reports of Clinical Trials
Lehman, E., DeYoung, J., Barzilay, R., & Wallace, B. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1371 · Cited by 101 · see citing works
Augmenting Scientific Papers with Just-in-Time, Position-Sensitive Definitions of Terms and Symbols
Head, A., Lo, K., Kang, D., Fok, R., Skjonsberg, S., Weld, D. S., & Hearst, M. A. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445648 · Cited by 91 · see citing works
BioASQ-QA: A manually curated corpus for Biomedical Question Answering
Krithara, A., Nentidis, A., Bougiatiotis, K., & Παλιούρας, Γ. (2023) · Scientific Data · doi:10.1038/s41597-023-02068-4 · Cited by 75 · see citing works
Measuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023) · Computational Linguistics · doi:10.1162/coli_a_00486 · Cited by 71 · see citing works
Jardim, P. S. J., Rose, C. J., Ames, H. M., Echavez, J. F. M., Van de Velde, S., & Muller, A. E. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01649-y · Cited by 68 · see citing works
Automatic Evaluation of Attribution by Large Language Models
Yue, X., Wang, B., Chen, Z., Zhang, K., Su, Y., & Sun, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.307 · Cited by 62 · see citing works
Facilitating Document Reading by Linking Text and Tables
Kim, D. H., Hoque, E., Kim, J., & Agrawala, M. (2018) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3242587.3242617 · Cited by 58 · see citing works
August, T., Wang, L. L., Bragg, J., Hearst, M. A., Head, A., & Lo, K. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589955 · Cited by 57 · see citing works
HEAD-QA: A Healthcare Dataset for Complex Reasoning
Vilares, D., & Gómez‐Rodríguez, C. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1092 · Cited by 50 · see citing works
Zhao, J., Huang, S., & Cole, J. M. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01259 · Cited by 50 · see citing works
Ponsard, A., Escalona, F., & Munzner, T. (2016) · doi:10.1145/2851581.2892334 · Cited by 44 · see citing works
Scim: Intelligent Skimming Support for Scientific Papers
Fok, R., Kambhamettu, H., Soldaini, L., Bragg, J., Lo, K., Hearst, M. A., Head, A., & Weld, D. S. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584034 · Cited by 43 · see citing works
Measuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2021) · arXiv · doi:10.48550/arxiv.2112.12870 · Cited by 34 · see citing works
Conversations with Documents: An Exploration of Document-Centered Assistance
ter Hoeve, M., Sim, R., Nouri, E., Fourney, A., de Rijke, M., & White, R. W. (2020) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3343413.3377971 · Cited by 32 · see citing works
Marvista: Exploring the Design of a Human-AI Collaborative News Reading Tool
Chen, X., Wu, C.-S., Murakhovs’ka, L., Laban, P., Niu, T., Liu, W., & Xiong, C. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3609331 · Cited by 29 · see citing works
A Dataset of Argumentative Dialogues on Scientific Papers
Ruggeri, F., Mesgar, M., & Gurevych, I. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2023.acl-long.425 · Cited by 28 · see citing works
CReBot: Exploring interactive question prompts for critical paper reading
Peng, Z., Liu, Y., Zhou, H., Xu, Z., & Ma, X. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102898 · Cited by 27 · see citing works
Trialstreamer: Mapping and Browsing Medical Evidence in Real-Time
Nye, B., Nenkova, A., Marshall, I. J., & Wallace, B. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.9 · Cited by 25 · see citing works
Lee, B., Savisaari, O., & Oulasvirta, A. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858299 · Cited by 18 · see citing works
Wang, X., Huang, R., Jin, Z., Fang, T., & Qu, H. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327153 · Cited by 18 · see citing works
Self-citationHuey, S. L., Konieczynski, E. M., Mehta, N. H., et al. (2023) · Nature Food · doi:10.1038/s43016-023-00874-y · Cited by 17 · see citing works
Self-citationKang, T., Sun, Y., Kim, J. H., Ta, C., Perotte, A., Schiffer, K., Wu, M., Zhao, Y., Moustafa-Fahmy, N., Peng, Y., et al. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad036 · Cited by 14 · see citing works
Lo, K., Shen, Z., Newman, B., Chang, J., Authur, R., Bransom, E., Candra, S., Chandrasekhar, Y., Huff, R., Kuehl, B., et al. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.45 · Cited by 14 · see citing works
Large Language Models in Health Care: Charting a Path Toward Accurate, Explainable, and Secure AI
Khullar, D., Wang, X., & Wang, F. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-08657-2 · Cited by 13 · see citing works
Self-citationZhang, S., Xu, H., Jia, Y., Wen, Y., Wang, D., Fu, L., Wang, X., & Zhou, C. (2023) · Geoscience Data Journal · doi:10.1002/gdj3.186 · Cited by 12 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 5
- Chau, D. H., A. Kittur, J. I. Hong, and C. Faloutsos. 2011. “ Apolo: Making Sense of Large Network Data by Combining Rich User Interaction and Machine Learning.” In Proceedings of the 2011 CHI Conference on Human Factors in Computing Systems, 167– 176. ACM.
- Es, S., J. James, L. Espinosa Anke, and S. Schockaert. 2024. “ RAGAs: Automated Evaluation of Retrieval Augmented Generation.” In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, edited by N. Aletras and O. De Clercq, 150– 158. Association for Computational Linguistics.
- Kang, H., J. C. Chang, Y. Kim, and A. Kittur. 2022. “ Threddy: An Interactive System for Personalized Thread‐Based Exploration and Organization of Scientific Literature.” In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, 1– 15. ACM.
- Kang, H. B., T. Wu, J. C. Chang, and A. Kittur. 2023. “ Synergi: A Mixed‐Initiative System for Scholarly Synthesis and Sensemaking.” In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1– 19. ACM.
- Lewis, P., E. Perez, A. Piktus, et al. 2020. “ Retrieval‐Augmented Generation for Knowledge‐Intensive Nlp Tasks.” Advances in Neural Information Processing Systems 33: 9459– 9474.
Cited by
Sorted by citations
Structured Schemas for Provenance-Rich, LLM-Assisted QSP Model Calibration
Eliason, J., & Popel, A. S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.05.709623
“LLMs can identify relevant litera-56 ture given mechanistic context, read scientific text, and extract information into structured tem-57 plates 14,15,16,17,18 .”
Structured Schemas for Provenance‐Rich, <scp>LLM</scp> ‐Assisted <scp>QSP</scp> Model Calibration
Eliason, J., & Popel, A. S. (2026) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.70306
- Authors
- Wang, X., Huey, S. L., Sheng, R., Mehta, S., & Wang, F.
- Affiliations
- Cornell University (US)
- Hong Kong University of Science and Technology (HK)
- Corresponding author
- Mehta, Saurabh
- Wang, Fei
- Venue
- Campbell Systematic Reviews: Better Evidence for a Better World
- Publisher
- SAGE Publications
- Published
- 2025-11-03
- Volume
- 21
- Issue
- 4
- Pages
- e70073-e70073
- ISSN
- 1891-1803 (ISSN-L)
- PMC
- PMC12581027
- OpenAlex
- W4415824463
- Cited by
- 13
- References
- 39
- Citations by year
- 2026: 9 · 2025: 4
- FWCI
- 18.16
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Campbell Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Institutes of Health
- National Institute of Child Health and Human Development
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (T32HD087137)
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- structured data extraction, scientific publications, large language models, table making, multimodal data integration, visual summary, semantic classification, human-AI collaboration, nutrition research, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
