The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
SQLizer: query synthesis from natural language
Open accessAbstract
This paper presents a new technique for automatically synthesizing SQL queries from natural language (NL). At the core of our technique is a new NL-based program synthesis methodology that combines semantic parsing techniques from the NLP community with type-directed program synthesis and automated program repair. Starting with a program sketch obtained using standard parsing techniques, our approach involves an iterative refinement loop that alternates between probabilistic type inhabitation and automated sketch repair. We use the proposed idea to build an end-to-end system called SQLIZER that can synthesize SQL queries from natural language. Our method is fully automated, works for any database without requiring additional customization, and does not require users to know the underlying database schema. We evaluate our approach on over 450 natural language queries concerning three different databases, namely MAS, IMDB, and YELP. Our experiments show that the desired query is ranked within the top 5 candidates in close to 90% of the cases and that SQLIZER outperforms NALIR, a state-of-the-art tool that won a best paper award at VLDB'14.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
Fundamentals of Database Systems
Elmasri, R. A., & Navathe, S. B. (1989) · Cited by 3,141
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 1,517
Empirical evaluation of the tarantula automatic fault-localization technique
Jones, J. A., & Harrold, M. J. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101949 · Cited by 1,238
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581397 · Cited by 904
Natural language interfaces to databases – an introduction
Androutsopoulos, I., Ritchie, G. D., & Thanisch, P. (1995) · Natural Language Engineering · doi:10.1017/s135132490000005x · Cited by 768
Automatically finding patches using genetic programming
Weimer, W., Nguyen, T., Le Goues, C., & Forrest, S. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070536 · Cited by 761
Learning to parse database queries using inductive logic programming
Zelle, J. M., & Mooney, R. J. (1996) · Cited by 642
Combinatorial sketching for finite programs
Solar-Lezama, A., Tancau, L., Bodik, R., Seshia, S., & Saraswat, V. (2006) · Proceedings of the 12th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1168857.1168907 · Cited by 635
Automating string processing in spreadsheets using input-output examples
Gulwani, S. (2011) · ACM SIGPLAN Notices · doi:10.1145/1925844.1926423 · Cited by 581
Developing a natural language interface to complex data
Hendrix, G. G., Sacerdoti, E. D., Sagalowicz, D., & Slocum, J. (1978) · ACM Transactions on Database Systems · doi:10.1145/320251.320253 · Cited by 534
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings of the 24th international conference on Software engineering - ICSE '02 · doi:10.1145/581396.581397 · Cited by 524
Towards a theory of natural language interfaces to databases
Popescu, A.-M., Etzioni, O., & Kautz, H. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604070 · Cited by 514
Automatic patch generation by learning correct code
Long, F., & Rinard, M. (2016) · Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages · doi:10.1145/2837614.2837617 · Cited by 481
Automating string processing in spreadsheets using input-output examples
Gulwani, S. (2011) · Proceedings of the 38th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/1926385.1926423 · Cited by 446
Constructing an interactive natural language interface for relational databases
Li, F., & Jagadish, H. V. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2735461.2735468 · Cited by 437
SemFix: program repair via semantic analysis
Nguyen, H. D. T., Qi, D., Roychoudhury, A., & Chandra, S. (2013) · doi:10.5555/2486788.2486890 · Cited by 398
Staged program repair with condition synthesis
Long, F., & Rinard, M. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786811 · Cited by 348
Program synthesis by sketching
Bodík, R., & Solar-Lezama, A. (2008) · Cited by 326
A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each
Le Goues, C., Dewey-Vogt, M., Forrest, S., & Weimer, W. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227211 · Cited by 326
From symptom to cause: localizing errors in counterexample traces
Ball, T., Naik, M., & Rajamani, S. K. (2003) · Proceedings of the 30th ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/604131.604140 · Cited by 294
An efficient easily adaptable system for interpreting natural language queries
Warren, D. H. D., & Pereira, F. C. N. (1982) · doi:10.5555/972942.972944 · Cited by 275
TEAM: An experiment in the design of transportable natural-language interfaces
Grosz, B. J., Appelt, D. E., Martin, P. A., & Pereira, F. C. N. (1987) · Artificial Intelligence · doi:10.1016/0004-3702(87)90011-7 · Cited by 264
Synthesizing data structure transformations from input-output examples
Feser, J. K., Chaudhuri, S., & Dillig, I. (2015) · Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2737924.2737977 · Cited by 251
Self-citationZloof, M. M. (1975) · doi:10.1145/1282480.1282482 · Cited by 240
Using string-kernels for learning semantic parsers
Kate, R. J., & Mooney, R. J. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220290 · Cited by 237
FlashMeta: a framework for inductive program synthesis
Polozov, O., & Gulwani, S. (2015) · Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications · doi:10.1145/2814270.2814310 · Cited by 225
FlashExtract: a framework for data extraction by examples
Le, V., & Gulwani, S. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594333 · Cited by 209
What Went Wrong: Explaining Counterexamples
Groce, A., & Visser, W. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-44829-2_8 · Cited by 204
Programming by sketching for bit-streaming programs
Solar-Lezama, A., Rabbah, R., Bodík, R., & Ebcioğlu, K. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065045 · Cited by 201
Programming by sketching for bit-streaming programs
Solar-Lezama, A., Rabbah, R., Bodík, R., & Ebcioğlu, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065045 · Cited by 195
Program synthesis from polymorphic refinement types
Polikarpova, N., Kuraj, I., & Solar-Lezama, A. (2016) · Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2908080.2908093 · Cited by 193
Popescu, A.-M., Armanasu, A., Etzioni, O., Ko, D., & Yates, A. (2004) · Proceedings of the 20th international conference on Computational Linguistics - COLING '04 · doi:10.3115/1220355.1220376 · Cited by 189
Learning to transform natural to formal languages
Kate, R. J., Wong, Y. W., & Mooney, R. J. (2005) · Cited by 188
Type-and-example-directed program synthesis
Osera, P.-M., & Zdancewic, S. (2015) · Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2737924.2738007 · Cited by 185
Cause clue clauses: error localization using maximum satisfiability
Jose, M., & Majumdar, R. (2011) · ACM SIGPLAN Notices · doi:10.1145/1993316.1993550 · Cited by 185
An extended model of natural logic
MacCartney, B., & Manning, C. D. (2009) · Proceedings of the Eighth International Conference on Computational Semantics - IWCS-8 '09 · doi:10.3115/1693756.1693772 · Cited by 184
Synthesizing highly expressive SQL queries from input-output examples
Wang, C., Cheung, A., & Bodik, R. (2017) · Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/3062341.3062365 · Cited by 178
Language to Code: Learning Semantic Parsers for If-This-Then-That Recipes
Quirk, C., Mooney, R., & Galley, M. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-1085 · Cited by 169
Seven Steps to Rendezvous with the Casual User.
Codd, E. F. (1974) · Cited by 165
Complete completion using types and weights
Gvero, T., Kunčak, V., Kuraj, I., & Piskač, R. (2013) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/2491956.2462192 · Cited by 146
Component-based synthesis of table consolidation and transformation tasks from examples
Feng, Y., Martins, R., Geffen, J. V., Dillig, I., & Chaudhuri, S. (2017) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3062341.3062351 · Cited by 146
Self-citationTran, Q. T., Chan, C.-Y., & Parthasarathy, S. (2009) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1559845.1559902 · Cited by 144
A fully statistical approach to natural language interfaces
Miller, S., Stallard, D. G., Bobrow, R., & Schwartz, R. M. (1996) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981863.981871 · Cited by 136
Component-based synthesis for complex APIs
Feng, Y., Martins, R., Wang, Y., Dillig, I., & Reps, T. (2016) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/3009837.3009851 · Cited by 129
Self-citationProgram synthesis using natural language
Desai, A., Gulwani, S., Hingorani, V., Jain, N., Karkare, A., Marron, M., Sailesh, R., & Roy, S. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884786 · Cited by 116
Jose, M., & Majumdar, R. (2011) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1993498.1993550 · Cited by 105
FlashRelate: extracting relational data from semi-structured spreadsheets using examples
Barowy, D. W., Gulwani, S., Hart, T., & Zorn, B. G. (2015) · Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2737924.2737952 · Cited by 100
Gulwani, S., & Marron, M. (2014) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2588555.2612177 · Cited by 97
Learning Semantic Grammars with Constructive nduct ive ogic
Zelle, J. M., & Mooney, R. J. (1993) · Cited by 90
Constructing a Generic Natural Language Interface for an XML Database
Li, Y., Yang, H., & Jagadish, H. V. (2006) · Lecture notes in computer science · doi:10.1007/11687238_44 · Cited by 87
Automated construction of database interfaces
Tang, L. R., & Mooney, R. J. (2000) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1117794.1117811 · Cited by 86
Le, V., Gulwani, S., & Su, Z. (2013) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2462456.2464443 · Cited by 83
Synthesizing data structure transformations from input-output examples
Feser, J., Chaudhuri, S., & Dillig, I. (2015) · ACM SIGPLAN Notices · doi:10.1145/2813885.2737977 · Cited by 81
Self-citationAutomatically synthesizing SQL queries from input-output examples
Zhang, S., & Sun, Y. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693082 · Cited by 78
MASQUE/SQL: an efficient and portable natural language query interface for relational databases
Androutsopoulos, I., Ritchie, G., & Thanisch, P. (1993) · Cited by 77
FlashExtract: a framework for data extraction by examples
Le, V., & Gulwani, S. (2014) · ACM SIGPLAN Notices · doi:10.1145/2666356.2594333 · Cited by 59
Bringing Machine Learning and Compositional Semantics Together
Liang, P., & Potts, C. (2015) · Annual Review of Linguistics · doi:10.1146/annurev-linguist-030514-125312 · Cited by 59
Bug-Assist: Assisting Fault Localization in ANSI-C Programs
Jose, M., & Majumdar, R. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22110-1_40 · Cited by 56
FlashMeta: a framework for inductive program synthesis
Polozov, O., & Gulwani, S. (2015) · ACM SIGPLAN Notices · doi:10.1145/2858965.2814310 · Cited by 48
Compositional program synthesis from natural language and examples
Raza, M., Gulwani, S., & Milić-Frayling, N. (2015) · International Conference on Artificial Intelligence · Cited by 47
Synthesizing transformations on hierarchically structured data
Yaghmazadeh, N., Klinger, C., Dillig, I., & Chaudhuri, S. (2016) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/2908080.2908088 · Cited by 46
Self-citation
Cited by
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1425 · Cited by 1,034
Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation
Guo, J., Zhan, Z., Gao, Y., Xiao, Y., Lou, J.-G., Liu, T., & Zhang, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1444 · Cited by 406
Coarse-to-Fine Decoding for Neural Semantic Parsing
Dong, L., & Lapata, M. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1068 · Cited by 374
SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
Xu, X., Liu, C., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04436 · Cited by 283
Li, J., Hui, B., Qu, G., Yang, J., Li, B., Li, B., Wang, B., Qin, B., Cao, R., Geng, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.03111 · Cited by 278
Metadata
- Authors
- Yaghmazadeh, N., Wang, Y., Dillig, I., & Dillig, T.
- Venue
- Proceedings of the ACM on Programming Languages
- Published
- 2017-10-12
- Volume
- 1
- Issue
- OOPSLA
- Pages
- 1-26
- ISSN
- 2475-1421 (ISSN-L)
- OpenAlex
- W2762513422
- Cited by
- 279
- References
- 62
- FWCI
- 16.40
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.