CoChat
Sign Up
Cited by 137Topic ModelingNatural Language Processing TechniquesSyntax, Semantics, Linguistic Variation

“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding

Open access
Zhou, Ben, Khashabi, Daniel, Ning, Qiang, Roth, Dan·2019·Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
DOI 10.18653/v1/d19-1332
Open this paper in CoChat — free accountOpen PDF

Abstract

Ben Zhou, Daniel Khashabi, Qiang Ning, Dan Roth. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/d19-1332

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 39 · Cited by 137

Research works printed but not linked: 33

  1. Lisa Bauer, Yicheng Wang, and Mohit Bansal. 2018. Commonsense for generative multi-hop question an- swering tasks. In Proceedings of the Conference on Empirical Methods for Natural Language Process- ing (EMNLP), pages 4220-4230.
  2. Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In ACL, Vancouver. ACL.
  3. Timothy Chklovski and Patrick Pantel. 2004. VerbO- cean: Mining the Web for Fine-Grained Semantic Verb Relations. In Proceedings of the Conference on Empirical Methods for Natural Language Pro- cessing (EMNLP), pages 33-40.
  4. Anne Cocos, Veronica Wharton, Ellie Pavlick, Mari- anna Apidianaki, and Chris Callison-Burch. 2018. Learning scalar adjective intensity from para- phrases. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process- ing, pages 1752-1762.
  5. Ernest Davis. 2014. Representations of commonsense knowledge. Morgan Kaufmann.
  6. Maxwell Forbes and Yejin Choi. 2017. Verb physics: Relative physical knowledge of actions and objects. In ACL, volume 1, pages 266-276.
  7. Mark Granroth-Wilding and Stephen Christopher Clark. 2016. What happens next? event predic- tion using a compositional neural network model. In ACL.
  8. Andrey Gusev, Nathanael Chambers, Pranav Khaitan, Divye Khilnani, Steven Bethard, and Dan Jurafsky. 2011. Using query patterns to learn the duration of events. In IWCS, pages 145-154. Association for Computational Linguistics.
  9. Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface: A challenge set for reading com- prehension over multiple sentences. In NAACL.
  10. Paul Kingsbury and Martha Palmer. 2002. From tree- bank to propbank. In LREC, pages 1989-1993.
  11. Zornitsa Kozareva and Eduard Hovy. 2011. Learning temporal information for states and events. In Fifth International Conference on Semantic Computing, pages 424-429. IEEE.
  12. Kenton Lee, Yoav Artzi, Jesse Dodge, and Luke Zettle- moyer. 2014. Context-dependent semantic parsing for time expressions. In ACL (1), pages 1437-1447.
  13. Artuur Leeuwenberg and Marie-Francine Moens. 2018. Temporal information extraction by predict- ing relative time-lines. Proceedings of the Confer- ence on Empirical Methods for Natural Language Processing (EMNLP).
  14. Zhongyang Li, Xiao Ding, and Ting Liu. 2018. Con- structing narrative event evolutionary graph for script event prediction. Proc. of the International Joint Conference on Artificial Intelligence (IJCAI).
  15. Elizabeth Merkhofer, John Henderson, David Bloom, Laura Strickhart, and Guido Zarrella. 2018. Mitre at semeval-2018 task 11: Commonsense reason- ing without commonsense knowledge. In SemEval, pages 1078-1082.
  16. Qiang Ning, Zhili Feng, and Dan Roth. 2017. A struc- tured learning approach to temporal relation extrac- tion. In Proceedings of the Conference on Em- pirical Methods in Natural Language Processing (EMNLP).
  17. Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018a. Joint reasoning for temporal and causal rela- tions. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  18. Qiang Ning, Hao Wu, Haoruo Peng, and Dan Roth. 2018b. Improving temporal relation extraction with a globally acquired statistical resource. In Proceed- ings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL).
  19. Qiang Ning, Hao Wu, and Dan Roth. 2018c. A multi- axis annotation scheme for event temporal relations. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics.
  20. Qiang Ning, Ben Zhou, Zhili Feng, Haoruo Peng, and Dan Roth. 2018d. CogCompTime: A tool for under- standing time in natural language. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP).
  21. Simon Ostermann, Michael Roth, Ashutosh Modi, Ste- fan Thater, and Manfred Pinkal. 2018. Semeval- 2018 task 11: Machine comprehension using com- monsense knowledge. In SemEval, pages 747-757.
  22. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 confer- ence on empirical methods in natural language pro- cessing (EMNLP), pages 1532-1543.
  23. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In NAACL.
  24. Hannah Rashkin, Maarten Sap, Emily Allaway, Noah A. Smith, and Yejin Choi. 2018. Event2Mind: Commonsense inference on events, intents, and re- actions. In Proc. of the Annual Meeting of the Asso- ciation of Computational Linguistics (ACL), pages 463-473.
  25. Melissa Roemmele, Cosmin Adrian Bejan, and An- drew S Gordon. 2011. Choice of plausible alterna- tives: An evaluation of commonsense causal reason- ing. In 2011 AAAI Spring Symposium Series.
  26. Lenhart Schubert. 2002. Can we derive general world knowledge from texts? In Proceedings of the second international conference on Human Language Tech- nology Research, pages 94-97.
  27. Jannik Strötgen and Michael Gertz. 2010. HeidelTime: High quality rule-based extraction and normaliza- tion of temporal expressions. In SemEval, pages 321-324. Association for Computational Linguis- tics.
  28. Niket Tandon, Bhavana Dalvi, Joel Grus, Wen-tau Yih, Antoine Bosselut, and Peter Clark. 2018. Reasoning about actions and state changes by injecting com- monsense knowledge. In Proceedings of the Con- ference on Empirical Methods for Natural Language Processing (EMNLP), pages 57-66.
  29. Alakananda Vempala, Eduardo Blanco, and Alexis Palmer. 2018. Determining event durations: Mod- els and error analysis. In NAACL, volume 2, pages 164-168.
  30. Jennifer Williams. 2012. Extracting fine-grained dura- tions for verbs from twitter. In Proceedings of ACL 2012 Student Research Workshop, pages 49-54. As- sociation for Computational Linguistics.
  31. Yiben Yang, Larry Birnbaum, Ji-Ping Wang, and Doug Downey. 2018. Extracting commonsense properties from embeddings with limited human guidance. In Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 644-649.
  32. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference. In EMNLP, pages 93-104.
  33. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Ben- jamin Van Durme. 2017. Ordinal common-sense in- ference. Transactions of the Association of Compu- tational Linguistics, 5(1):379-395.

Authors
Zhou, B., Khashabi, D., Ning, Q., & Roth, D.
Affiliations
  • University of Pennsylvania (US)
  • Allen Institute for Artificial Intelligence (US)
ORCID
Venue
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Publisher
Association for Computational Linguistics
Published
2019
Pages
3361-3367
DOI
10.18653/v1/d19-1332
OpenAlex
W2971236147
Cited by
137
References
39
Citations by year
2026: 15 · 2025: 8 · 2024: 6 · 2023: 26 · 2022: 22 · 2021: 33 · 2020: 24 · 2019: 3
FWCI
7.84
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
Funders
  • U.S. Department of Defense
  • Allen Institute for Artificial Intelligence
  • Defense Advanced Research Projects Agency (HR0011-15-C-0113)
  • Advanced Research Projects Agency
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
Keywords
temporal reasoning, commonsense knowledge, temporal relations, event duration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review