CoChat
Sign Up
Cited by 78Artificial Intelligence in LawTopic ModelingSentiment Analysis and Opinion Mining

Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases

Open access
Chalkidis, Ilias, Fergadiotis, Manos, Tsarapatsanis, Dimitrios, Aletras, Nikolaos, Androutsopoulos, Ion, Malakasiotis, Prodromos·2021·Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
DOI 10.18653/v1/2021.naacl-main.22
Open this paper in CoChat — free accountOpen PDF

Abstract

Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, Prodromos Malakasiotis. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2021.naacl-main.22

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Training Transformers with longer sequences with FlashAttention improves performance on the MIMIC-III and ECtHR datasets.

    “Training Transformers with longer sequences with FlashAttention improves performance on the MIMIC-III [47] and ECtHR [6, 7] datasets.”

    From FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) · cites this paper as [7]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 50 · Cited by 78

Research works printed but not linked: 35

  1. David Alvarez-Melis and Tommi Jaakkola. 2017. A causal framework for explaining the predictions of black-box sequence-to-sequence models. In Pro- ceedings of the 2017 Conference on Empirical Meth- ods in Natural Language Processing, pages 412- 421, Copenhagen, Denmark. Association for Com- putational Linguistics. · DOI
  2. Woodrow Barfield. 2020. The Cambridge Handbook of the Law of Algorithms. Cambridge Law Handbooks. Cambridge University Press. · DOI
  3. Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2019. Interpretable neural predictions with differentiable binary variables. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, pages 2963-2977, Florence, Italy. Associa- tion for Computational Linguistics. · DOI
  4. Ilias Chalkidis, Emmanouil Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androut- sopoulos. 2020. Legal-bert: The muppets straight out of law school. Finding of EMNLP.
  5. Shiyu Chang, Yang Zhang, Mo Yu, and Tommi S. Jaakkola. 2019. A game theoretic approach to class- wise selective rationalization. In Advances in Neural Information Processing Systems (NeurIPS), Vancou- ver, Canada.
  6. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT's attention. In Pro- ceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276-286, Florence, Italy. Association for Computational Linguistics. · DOI
  7. Madeleine Clare Elish, William Isaac, and Richard Zemel, editors. 2021. Proceedings of the Confer- ence on Fairness, Accountability, and Transparency (FAccT 2021). Association for Computing Machin- ery, Online.
  8. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative ad- versarial nets. In Proceedings of the 27th Interna- tional Conference on Neural Information Processing Systems, page 2672-2680, Montreal, Canada.
  9. David John Harris. 2018. Harris, O'Boyle, and War- brick : Law of the European Convention on Hu- man Rights, 4rd edition. edition. Oxford University Press.
  10. Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, and Maosong Sun. 2018. Few-shot charge prediction with discriminative legal attributes. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 487-498.
  11. Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stan- forth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2020. Reducing sentiment bias in language models via counterfac- tual evaluation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 65-83, Online. Association for Computational Lin- guistics. · DOI · Paper page
  12. Alon Jacovi and Yoav Goldberg. 2020. Towards faith- fully interpretable NLP systems: How should we de- fine and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 4198-4205, Online. As- sociation for Computational Linguistics. · DOI · Paper page
  13. Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. In Proceedings of the Annual Con- ference of the North American Chapter of the Associ- ation for Computational Linguistics (NAACL-HLT).
  14. Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, and By- ron C. Wallace. 2020. Learning to faithfully rational- ize by construction. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 4459-4473, Online. Association for Computational Linguistics. · DOI
  15. Daniel Martin Katz. 2012. Quantitative legal prediction-or-how I learned to stop worrying and start preparing for the data-driven future of the legal services industry. Emory Law Journal, 62:909.
  16. Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface:a challenge set for reading com- prehension over multiple sentences. In Proceedings of North American Chapter of the Association for Computational Linguistics (NAACL).
  17. Günter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter. 2017. Self-normalizing neural networks. In 31th Int. Conf. on Neural Infor- mation Processing Systems.
  18. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP), pages 4365-4374, Hong Kong, China. Association for Computational Linguistics. · DOI
  19. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Nat- ural Language Processing, pages 107-117, Austin, Texas.
  20. Daniel Locke and Guido Zuccon. 2018. A test collec- tion for evaluating legal case law search. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR '18, page 1261-1264, New York, NY, USA. Association for Computing Machinery. · DOI
  21. Christos Louizos and Max Welling. 2017. Multiplica- tive normalizing flows for variational bayesian neu- ral networks. In Proceedings of the International Conference on Machine Learning (ICML 2017), Sydney, Australia.
  22. Bingfeng Luo, Yansong Feng, Jianbo Xu, Xiang Zhang, and Dongyan Zhao. 2017. Learning to predict charges for criminal cases with legal basis. In Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing (Conference on Em- pirical Methods in Natural Language Processing (EMNLP)), pages 2727-2736.
  23. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2009. Introduction to Information Retrieval. Cambridge University Press.
  24. Masha Medvedeva, Michel Vols, and Martijn Wieling. 2018. Judicial decisions of the European Court of Human Rights: Looking into the crystal ball. In Proceedings of the Conference on Empirical Legal Studies.
  25. W. James Murdoch, Peter J. Liu, and Bin Yu. 2018. Be- yond word importance: Contextual decomposition to extract interactions from lstms. In Proceedings of International Conference on Learning Represen- tations (ICLR).
  26. Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should I trust you?": Ex- plaining the predictions of any classifier. CoRR, abs/1602.04938. · arXiv
  27. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems (NeurIPS).
  28. Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Con- ference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- IJCNLP), pages 11-20, Hong Kong, China. Associ- ation for Computational Linguistics. · DOI · Paper page
  29. Ronald J. Williams. 1992. Simple statistical gradient- following algorithms for connectionist reinforce- ment learning. Machine Learning, 8(3-4):229-256. · DOI
  30. Liu Yang, Mingyang Zhang, Cheng Li, Michael Ben- dersky, and Marc Najork. 2020. Beyond 512 to- kens: Siamese multi-depth transformer-based hier- archical encoder for long-form document matching. In Proceedings of the 29th ACM International Con- ference on Information & Knowledge Management, page 1725-1734, New York, NY, USA. Association for Computing Machinery. · DOI
  31. Zichao Yang et al. 2016. Hierarchical attention net- works for document classification. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Lin- guistics (NAACL-HLT), pages 1480-1489.
  32. Mo Yu, Shiyu Chang, Yang Zhang, and Tommi Jaakkola. 2019. Rethinking cooperative rationaliza- tion: Introspective extraction and complement con- trol. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP), pages 4094-4103, Hong Kong, China. Association for Computational Linguistics. · DOI
  33. Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using "annotator rationales" to improve machine learning for text categorization. In Human Lan- guage Technologies 2007: The Conference of the North American Chapter of the Association for Com- putational Linguistics; Proceedings of the Main Conference, pages 260-267, Rochester, New York. Association for Computational Linguistics.
  34. Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. How does NLP benefit legal system: A summary of legal artificial intelligence. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 5218-5230, Online. Association for Computational Linguistics. · DOI
  35. Haoxi Zhong, Guo Zhipeng, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, and Maosong Sun. 2018. Le- gal judgment prediction via topological learning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3540-3549.

Authors
Chalkidis, I., Fergadiotis, M., Tsarapatsanis, D., Aletras, N., Androutsopoulos, I., & Malakasiotis, P.
Affiliations
  • National Centre of Scientific Research "Demokritos" (GR)
  • Athens University of Economics and Business (GR)
ORCID
Venue
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Publisher
Association for Computational Linguistics
Published
2021
Pages
226-241
DOI
10.18653/v1/2021.naacl-main.22
OpenAlex
W3136888420
Cited by
78
References
50
Citations by year
2026: 7 · 2025: 11 · 2024: 16 · 2023: 26 · 2022: 9 · 2021: 9
FWCI
94.63
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
Funders
  • European Commission
  • Engineering and Physical Sciences Research Council (EP/V055712/1)
  • CHIST-ERA
Topics
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
regularization, legal documents

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review