CoChat
Sign Up
PreprintCited by 144Natural Language Processing TechniquesSpeech and dialogue systemsTopic Modeling

The DOI matches its DataCite record.

NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System

Open access
Lin, Xi Victoria, Wang, Chenglong, Zettlemoyer, Luke, Ernst, Michael D.·2018·arXiv
DOI 10.48550/arxiv.1802.08979arXiv 1802.08979
Open this paper in CoChat — free accountOpen PDF

Abstract

We present new data and semantic parsing methods for the problem of mapping English sentences to Bash commands (NL2Bash). Our long-term goal is to enable any user to perform operations such as file manipulation, search, and application-specific scripting by simply stating their goals in English. We take a first step in this domain, by providing a new dataset of challenging but commonly used Bash commands and expert-written English descriptions, along with baseline methods to establish performance levels on this task.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1802.08979

Cites

Statistics

Figures

7 Figure 1 : Figure 1: Top 50 most frequent bash utilities in the dataset with their frequencies in log scale. U1 and U2 at the bottom of the circle denote the utilities basename and readlink.
7 Figure 1 : Figure 1: Top 50 most frequent bash utilities in the dataset with their frequencies in log scale. U1 and U2 at the bottom of the circle denote the utilities basename and readlink.CC-BY-NC · From the PDF
Figure 2 : Figure 2: Error overlap of ST-CopyNet and Tellina. The number denotes the percentage out of the 100 dev samples.
Figure 2 : Figure 2: Error overlap of ST-CopyNet and Tellina. The number denotes the percentage out of the 100 dev samples.CC-BY-NC · From the PDF
Figure 3 : Figure 3: Number of error instances in each error classes of ST-CopyNet and Tellina. The classes marked with v are unique to the pipeline system.
Figure 3 : Figure 3: Number of error instances in each error classes of ST-CopyNet and Tellina. The classes marked with v are unique to the pipeline system.CC-BY-NC · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 30

  1. Gu, J., Lu, Z., Li, H., and Li, V. O. K. (2016). Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the 54th Annual Meeting of the Associ- ation for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers.
  2. Guu, K., Pasupat, P., Liu, E. Z., and Liang, P. (2017). From language to programs: Bridging reinforcement learning and maximum marginal likelihood. In Proceedings of the 55th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2017, Vancouver, Canada, July 30 -August 4, Volume 1: Long Papers, pages 1051-1062.
  3. Haas, C. and Riezler, S. (2016). A corpus and semantic parser for multilingual natural language querying of open- streetmap. In Kevin Knight, et al., editors, NAACL HLT 2016, The 2016 Conference of the North American Chap- ter of the Association for Computational Linguistics: Hu- man Language Technologies, San Diego California, USA, June 12-17, 2016, pages 740-750. The Association for Computational Linguistics.
  4. Iyer, S., Konstas, I., Cheung, A., and Zettlemoyer, L. (2016). Summarizing source code using a neural attention model. In Proceedings of the 54th Annual Meeting of the Associ- ation for Computational Linguistics, ACL 2016, August 7-12, 2016, Volume 1: Long Papers, pages 2073-2083, Berlin, Germany.
  5. Iyer, S., Konstas, I., Cheung, A., Krishnamurthy, J., and Zettlemoyer, L. (2017). Learning a neural semantic parser from user feedback. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics, ACL 2017, Vancouver, Canada, July 30 -August 4, Volume 1: Long Papers, pages 963-973.
  6. Jain, L. C. and Medsker, L. R. (1999). Recurrent Neural Networks: Design and Applications. CRC Press, Inc., Boca Raton, FL, USA, 1st edition.
  7. Kushman, N. and Barzilay, R. (2013). Using semantic uni- fication to generate regular expressions from natural lan- guage. In Lucy Vanderwende, et al., editors, Human Lan- guage Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 9-14, 2013, pages 826-836, Westin Peachtree Plaza Hotel, Atlanta, Georgia, USA. The Asso- ciation for Computational Linguistics.
  8. Kwiatkowski, T., Choi, E., Artzi, Y., and Zettlemoyer, L. (2013). Scaling semantic parsers with on-the-fly ontol- ogy matching. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1545-1556, Seattle, Washington, USA, October. Association for Computational Linguistics.
  9. Ling, W., Blunsom, P., Grefenstette, E., Hermann, K. M., Kociský, T., Wang, F., and Senior, A. (2016). Latent pre- dictor networks for code generation. In Proceedings of the 54th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers.
  10. Locascio, N., Narasimhan, K., DeLeon, E., Kushman, N., and Barzilay, R. (2016). Neural generation of regu- lar expressions from natural language with minimal do- main knowledge. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP, November 1-4, 2016, pages 1918-1923, Austin, Texas, USA.
  11. Long, R., Pasupat, P., and Liang, P. (2016). Simpler context- dependent logical forms via model projections. In Pro- ceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers.
  12. Nie, P., Li, J. J., Khurshid, S., Mooney, R., and Gligoric, M. (2018). Natural language processing and program analy- sis for supporting todo comments as software evolves. In Proceedings of the AAAI Workshop of Statistical Model- ing of Natural Software Corpora.
  13. Oda, Y., Fudaba, H., Neubig, G., Hata, H., Sakti, S., Toda, T., and Nakamura, S. (2015). Learning to generate pseudo-code from source code using statistical machine translation (T). In Myra B. Cohen, et al., editors, 30th IEEE/ACM International Conference on Automated Soft- ware Engineering, ASE 2015, Lincoln, NE, USA, Novem- ber 9-13, 2015, pages 574-584. IEEE Computer Society.
  14. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, pages 311-318, Stroudsburg, PA, USA. Association for Computational Linguistics.
  15. Polosukhin, I. and Skidanov, A. (2018). Neural Program Search: Solving Programming Tasks from Description and Examples. ArXiv e-prints, February.
  16. Popescu, A.-M., Etzioni, O., and Kautz, H. (2003). Towards a theory of natural language interfaces to databases. In Proceedings of the 8th International Conference on In- telligent User Interfaces, IUI '03, pages 149-157, New York, NY, USA. ACM.
  17. Quirk, C., Mooney, R. J., and Galley, M. (2015). Language to code: Learning semantic parsers for if-this-then-that recipes. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Volume 1: Long Papers, pages 878-888, Beijing, China. The Association for Computer Linguistics.
  18. Sammet, J. E. (1966). The use of english as a programming language. Communications of the ACM, 9(3):228-230.
  19. Schuster, M. and Paliwal, K. (1997). Bidirectional recurrent neural networks. Trans. Sig. Proc., 45(11):2673-2681, November.
  20. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Infor- mation Processing Systems, NIPS'14, pages 3104-3112, Cambridge, MA, USA. MIT Press.
  21. Tang, L. R. and Mooney, R. J. (2001). Using multiple clause constructors in inductive logic programming for semantic parsing. In Luc De Raedt et al., editors, Machine Learning: EMCL 2001, 12th European Conference on Machine Learning, Freiburg, Germany, September 5-7, 2001, Proceedings, volume 2167 of Lecture Notes in Computer Science, pages 466-477. Springer.
  22. Wilensky, R., Arens, Y., and Chin, D. (1984). Talking to unix in english: An overview of uc. Commun. ACM, 27(6):574-593, June.
  23. Wilensky, R., Chin, D. N., Luria, M., Martin, J., Mayfield, J., and Wu, D. (1988). The berkeley unix consultant project. Comput. Linguist., 14(4):35-84, December.
  24. Yao, Z., Weld, D., Chen, W.-P., and Sun, H. (2018). Staqc: A systematically mined question-code dataset from stack overflow. In Proceedings of the 27th International Con- ference on World Wide Web, WWW 2018, Lyon, France, April 23 -27, 2018.
  25. Yih, W., Richardson, M., Meek, C., Chang, M., and Suh, J. (2016). The value of semantic parse labeling for knowl- edge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers.
  26. Yin, P. and Neubig, G. (2017). A syntactic neural model for general-purpose code generation. In Proceedings of the 55th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2017, Vancouver, Canada, July 30 -August 4, Volume 1: Long Papers, pages 440-450.
  27. Zelle, J. M. and Mooney, R. J. (1996). Learning to parse database queries using inductive logic programming. In Proceedings of the Thirteenth National Conference on Artificial Intelligence -Volume 2, AAAI'96, pages 1050- 1055. AAAI Press.
  28. Zettlemoyer, L. S. and Collins, M. (2005). Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, UAI'05, pages 658-666, Arlington, Virginia, United States. AUAI Press.
  29. Zhong, V., Xiong, C., and Socher, R. (2017). Seq2sql: Gen- erating structured queries from natural language using re- inforcement learning. arXiv preprint arXiv:1709.00103. · arXiv
  30. Zhong, Z., Guo, J., Yang, W., Xie, T., Lou, J.-G., Liu, T., and Zhang, D. (2018). Generating regular expressions from natural language specifications: Are we there yet? In Proceedings of the AAAI Workshop of Statistical Modeling of Natural Software Corpora.

Cited by

Sorted by citations

Metadata

Authors
Lin, X. V., Wang, C., Zettlemoyer, L., & Ernst, M. D.
Affiliations
University of Washington (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
DOI
10.48550/arxiv.1802.08979
arXiv
1802.08979
OpenAlex
W2788109215
Cited by
144
References
0
Citations by year
2026: 26 · 2025: 12 · 2024: 11 · 2023: 16 · 2022: 22 · 2021: 23 · 2020: 16 · 2019: 13 · 2018: 5
FWCI
5.15
Type
conference paper
Language
en
Open access
Licence
Locations
  • Language Resources and Evaluation Conference (LREC) (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • Zenodo (CERN European Organization for Nuclear Research)
Funders
  • National Science Foundation (1562364)
  • Tencent
  • Defense Advanced Research Projects Agency (FA8750-13-2-0019)
  • Army Research Office
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
semantic parsing, Linux, command-line interface, corpus construction, file manipulation, scripting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review