Explainable artificial intelligence, lawyer's perspective
Abstract
Explainable artificial intelligence (XAI) is a research direction that was already put under scrutiny, in particular in the AI&Law community. Whilst there were notable developments in the area of (general, not necessarily legal) XAI, user experience studies regarding such methods, as well as more general studies pertaining to the concept of explainability among the users are still lagging behind. This paper firstly, assesses the performance of different explainability methods (Grad-CAM, LIME, SHAP), in explaining the predictions for a legal text classification problem; those explanations were then judged by legal professionals according to their accuracy. Secondly, the same respondents were asked to give their opinion on the desired qualities of (explainable) artificial intelligence (AI) legal decision system and to present their general understanding of the term XAI. This part was treated as a pilot study for a more pronounced one regarding the lawyer's position on AI, and XAI in particular.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tax explanations must account for the normative dimension of tax-related decisions, which requires new techniques that are still under development.
“Hence, tax explanations also need to account for the normative dimension of tax-related decisions, a task that requires new techniques still under development.28”
From Towards eXplainable Artificial Intelligence (XAI) in Tax Law: The Need for a Minimum Legal Standard (2022) · cites this paper as [28]
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · see citing works
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Proceedings of the 21st International Conference on Neural Information Processing Systems
Koller, D., Schuurmans, D., Bengio, Y., & Bottou, L. (2008) · Cited by 4,461 · see citing works
Consistent Individualized Feature Attribution for Tree Ensembles
Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430 · see citing works
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375830 · Cited by 900 · see citing works
Evaluating XAI: A comparison of rule-based and example-based explanations
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393 · see citing works
Kenny, E. M., Ford, C., Quinn, M., & Keane, M. T. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103459 · Cited by 211 · see citing works
A Misdirected Principle with a Catch: Explicability for AI
Robbins, S. (2019) · Minds and Machines · doi:10.1007/s11023-019-09509-3 · Cited by 175 · see citing works
AI and Law: A fruitful synergy
Rissland, E. L., Ashley, K. D., & Loui, R. P. (2003) · Artificial Intelligence · doi:10.1016/s0004-3702(03)00122-x · Cited by 136 · see citing works
Keane, M. T., & Kenny, E. M. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-29249-2_11 · Cited by 84 · see citing works
The Judicial Demand for Explainable Artificial IntelligencePreprint
Deeks, A. S. (2019) · SSRN Electronic Journal · Cited by 74 · see citing works
Mach, N. (2011) · DIAL (Catholic University of Leuven) · doi:10.1007/978-94-007-1497-7 · Cited by 45 · see citing works
SHAP values for Explaining CNN-based Text Classification ModelsPreprint
Zhao, W., Joshi, T., Nair, V. N., & Sudjianto, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11825 · Cited by 29 · see citing works
Semantic types for computational legal reasoning
Walker, V. R., Han, J. H., Ni, X., & Yoseda, K. (2017) · International Conference on Artificial Intelligence and Law · doi:10.1145/3086512.3086535 · Cited by 19 · see citing works
Governmental Transparency in the Era of Artificial Intelligence.
van Engers, T. M., & de Vries, D. M. (2019) · UvA-DARE (University of Amsterdam) · doi:10.3233/faia190304 · Cited by 5 · see citing works
Surrogate Locally-Interpretable Models with Supervised Machine Learning AlgorithmsPreprint
Hu, L., Chen, J., Nair, V. N., & Sudjianto, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14528 · Cited by 5 · see citing works
On Requirements and Design Criteria for Explainability in Legal AI
van Otterlo, M., & Atzmueller, M. (2018) · Tilburg University Research Portal · Cited by 4 · see citing works
Flagged first, then by citations
2 of the 24 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Haque, A. B., Islam, A. K. M. N., & Mikalef, P. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122120 · Cited by 280 · see citing works
Survey on Explainable AI: From Approaches, Limitations and Applications Aspects
Yang, W., Wei, Y.-C., Wei, H., Chen, Y., Huang, G., Li, X., Li, R., Yao, N., Wang, X., Gu, X., et al. (2023) · Human-Centric Intelligent Systems · doi:10.1007/s44230-023-00038-y · Cited by 275 · see citing works
XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process
Clement, T., Kemmerzell, N., Abdelaal, M., & Amberg, M. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010006 · Cited by 131 · see citing works
A survey on legal question–answering systems
Martínez-Gil, J. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100552 · Cited by 64 · see citing works
Kawakura, S., Hirafuji, M., Ninomiya, S., & Shibasaki, R. (2022) · European Journal of Agriculture and Food Sciences · doi:10.24018/ejfood.2022.4.6.348 · Cited by 49 · see citing works
Al-Ansari, N., Al‐Thani, D., & Al-Mansoori, R. S. (2024) · Human Behavior and Emerging Technologies · doi:10.1155/2024/4628855 · Cited by 52 · see citing works
Jiang, C., & Yang, X. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595170 · Cited by 45 · see citing works
Kharchenko, V. S., Fesenko, H. V., & Illiashenko, O. A. (2022) · Sensors · doi:10.3390/s22134865 · Cited by 49 · see citing works
Explainable AI Frameworks: Navigating the Present Challenges and Unveiling Innovative Applications
Sharma, N. A., Chand, R. R., Buksh, Z., Ali, A. B. M. S., Hanif, A., & Beheshti, A. (2024) · Algorithms · doi:10.3390/a17060227 · Cited by 49 · see citing works
Towards eXplainable Artificial Intelligence (XAI) in Tax Law: The Need for a Minimum Legal Standard
Kuźniacki, B., Almada, M., Tyliński, K., Górski, L., Winogradska, B., & Zeldenrust, R. (2022) · World Tax Journal · doi:10.59403/2yhh9pa · Cited by 31 · see citing works
Self-citationExploring explainable AI in the tax domain
Górski, Ł., Kuźniacki, B., Almada, M., Tyliński, K., Calvo, M., Asnaghi, P. M., Almada, L., Iñiguez, H. I., Rubianes, F., Pera, O., et al. (2024) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09395-w · Cited by 23 · see citing works
Self-citationAn Explainable AI Model for Hate Speech Detection on Indonesian Twitter
Ibrahim, M. A., Arifin, S., Yudistira, I. G. A. A., Nariswari, R., Abdillah, A. A., Murnaka, N. P., & Prasetyo, P. W. (2022) · CommIT (Communication and Information Technology) Journal · doi:10.21512/commit.v16i2.8343 · Cited by 15 · see citing works
Legal IR and NLP: The History, Challenges, and State-of-the-Art
Ganguly, D., Conrad, J. G., Ghosh, K., Ghosh, S., Goyal, P., Bhattacharya, P., Nigam, S. K., & Paul, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_34 · Cited by 17 · see citing works
Boosting court judgment prediction and explanation using legal entities
Benedetto, I., Koudounas, A., Vaiani, L., Pastor, E., Cagliero, L., Tarasconi, F., & Baralis, E. (2024) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09397-8 · Cited by 17 · see citing works
Requirements for Tax XAI Under Constitutional Principles and Human Rights
Kuzniacki, B., Almada, M., Tyliński, K., & Górski, Ł. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-15565-9_14 · Cited by 14 · see citing works
Self-citationRationale Discovery and Explainable AI
Steging, C., Renooij, S., & Verheij, B. (2021) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia210341 · Cited by 14 · see citing works
T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformers
Ntrougkas, M. V., Gkalelis, N., & Mezaris, V. (2024) · IEEE Access · doi:10.1109/access.2024.3405788 · Cited by 12 · see citing works
Explainable AI for online disinformation detection: Insights from a design science research project
Bezzaoui, I., Stein, C., Weinhardt, C., & Fegert, J. (2025) · Electronic Markets · doi:10.1007/s12525-025-00799-3 · Cited by 10 · see citing works
González‐González, J., de Arriba-Pérez, F., García-Méndez, S., Busto-Castiñeira, A., & González-Castaño, F. J. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101634 · Cited by 15 · see citing works
Prediction of Turkish Constitutional Court Decisions with Explainable Artificial Intelligence
TURAN, T., Küçüksille, E. U., & Kemaloğlu, N. (2023) · Bilge International Journal of Science and Technology Research · doi:10.30516/bilgesci.1317525 · Cited by 8 · see citing works
Evaluating Explanation Correctness in Legal Decision Making
Luo, C. F., Bhambhoria, R. V., Dahan, S., & Zhu, X. (2022) · doi:10.21428/594757db.8718dc8b · Cited by 6 · see citing works
Prototype-Based Interpretability for Legal Citation Prediction
Luo, C. F., Bhambhoria, R. V., Dahan, S., & Zhu, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.301 · Cited by 7 · see citing works
Effects of XAI on Legal Process
Nielsen, A., Skylaki, S., Norkutė, M., & Stremitzer, A. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595128 · Cited by 7 · see citing works
Lee, J. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09454-w · Cited by 6 · see citing works
A Vision on What Explanations of Autonomous Systems are of Interest to Lawyers
Buiten, M. C., Dennis, L. A., & Schwammberger, M. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00062 · Cited by 6 · see citing works
A Systematic Review on Explainable AI in Legal Domain
Joshi, M. G. (2024) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2024.61736 · Cited by 5 · see citing works
Porębski, A. (2023) · Research Handbook on Law and Technology · doi:10.4337/9781803921327.00037 · Cited by 4 · see citing works
Benedetto, I., Koudounas, A., Vaiani, L., Pastor, E., Baralis, E., Cagliero, L., & Tarasconi, F. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.194 · Cited by 4 · see citing works
Unveiling the Depths of Explainable AI
Khan, W., & Ishrat, M. (2024) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/979-8-3693-0968-1.ch004 · Cited by 4 · see citing works
Explainable Artificial Intelligence for Agile Mediation Propensity Assessment
Collini, E., Nesi, P., Raffaelli, C., & Scandiffio, F. (2024) · IEEE Access · doi:10.1109/access.2024.3375766 · Cited by 4 · see citing works
Al-Ansari, N., Al‐Thani, D., & Bahameish, M. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0567-5_6 · Cited by 1 · see citing works
Attorneys and AI: How Lawyers Use Artificial Intelligence and Analyze Its Impacts
Schieber, E. A. G., Kite, N., Hall, M. I., Turner, C., & Schlesinger, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757554 · Cited by 1 · see citing works
Motta, I., Lima, M. J., Ambrosio, T., Celecia, A., Mangeth, A. L., Frajhof, I. Z., & Cury, C. F. (2023) · Linköping electronic conference proceedings · doi:10.3384/ecp203061 · Cited by 1 · see citing works
Xu, G., & Wu, L. (2023) · International Conference on Algorithms, High Performance Computing and Artificial Intelligence (AHPCAI) · doi:10.1117/12.3011513 · Cited by 1 · see citing works
Al-Ansari, N., Al‐Thani, D., & Bahameish, M. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101083
Machkour, B., & Abriane, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-01536-5_7
Towards eXplainable Artificial Intelligence (XAI) in tax law : the need for a minimum legal standard
ALMADA, M., ALMADA, M., TYLIŃSKI, K., GÓRSKI, Ł., WINOGRADSKA, B., & ZELDENRUST, R. (2022) · Cadmus - EUI Research Repository (European University Institute)
Self-citationExplainable AI Application in Sentence Interpretation in Brazilian Law
Silva, F. A. C., & Rodrigues, C. M. D. O. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-12882-9_24
Méthodes d'apprentissage automatique pour l'analyse de corpus jurisprudentiels
Condevaux, C. (2021) · theses.fr (ABES) · doi:10.70675/a07b8042z06e3z4d21z97fbz209b3ca1618e
Explainable AI techniques for legal sentence classification using transformers
Sheik, R., Kuppachi, A., & Nirmala, S. J. (2026) · Sadhana · doi:10.1007/s12046-026-03299-y
- Authors
- Górski, Ł., & Ramakrishna, S.
- Affiliations
- Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw (PL)
- Freie Universität Berlin (DE)
- Venue
- Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law
- Publisher
- ACM
- Published
- 2021-06-21
- Pages
- 60-68
- OpenAlex
- W3186367509
- Cited by
- 40
- References
- 24
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 9 · 2023: 14 · 2022: 8 · 2021: 2
- FWCI
- 3.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Artificial Intelligence and Law (publishedVersion)
- Funders
- Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw (GR81-14)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- explainable AI, lawyers, lawyers' perceptions, legal text classification, Grad-CAM, lime, SHAP, explainability evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
