Semi-Supervised Methods for Explainable Legal Prediction
Abstract
Legal decision-support systems have the potential to improve access to justice, administrative efficiency, and judicial consistency, but broad adoption of such systems is contingent on development of technologies with low knowledge-engineering, validation, and maintenance costs. This paper describes two approaches to an important form of legal decision support---explainable outcome prediction---that obviate both annotation of an entire decision corpus and manual processing of new cases. The first approach, which uses an Attention Network for prediction and attention weights to highlight salient case text, was shown to be capable of predicting decisions, but attention-weight-based text highlighting did not demonstrably improve human decision speed or accuracy in an evaluation with 61 human subjects. The second approach, termed SCALE (Semi-supervised Case Annotation for Legal Explanations), exploits structural and semantic regularities in case corpora to identify textual patterns that have both predictable relationships to case decisions and explanatory value.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It would in all likelihood be difficult to ensure transparency of a machine-learning ADM system so as to produce a relevant explanation to the individual of how, and on the basis of what information, the ADM had arrived at a certain conclusion.
“it would in all likelihood be difficult to ensure transparency of a machinelearning ADM system so as to produce a relevant explanation to the individual of how, and on the basis of what information, the ADM had arrived at a certain conclusion (see Branting et al. 2019, 22”
From Administrative due process when using automated decision-making in public administration: some notes from a Finnish perspective (2021) · cites this paper as Branting et al. 2019, 22
A rule-based ADM solution appears to be the only option when ADM systems are designed for administrative decision-making.
“a rule-based ADM solution appears to be the only option when ADM systems are designed for administrative decision-making”
From Administrative due process when using automated decision-making in public administration: some notes from a Finnish perspective (2021) · cites this paper as Branting et al. 2019
Decision prediction is not appropriate for completely autonomous processes because denial of benefits by an automated process raises significant due process issues, and prediction accuracy is limited by the absence of explicit modeling of legal rules or issues.
“'decision prediction is not appropriate for completely autonomous processes' because '[d] enial of benefits by an automated process, no matter how accurate, raises significant due process issues, and in any event prediction accuracy is limited in this paradigm by the absence of explicit modeling of legal rules or issues'”
From Administrative due process when using automated decision-making in public administration: some notes from a Finnish perspective (2021) · cites this paper as Branting et al. 2019
Cites
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 4,874 · see citing works
A maximum entropy approach to natural language processing
Berger, A. L., Pietra, V. J. D., & Pietra, S. A. D. (1996) · doi:10.5555/234285.234289 · Cited by 3,083 · see citing works
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works
Αλέτρας, Ν., Tsarapatsanis, D., Preoțiuc-Pietro, D., & Lampos, V. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.93 · Cited by 813 · see citing works
Argumentation in artificial intelligence
Bench-Capon, T. J. M., & Dunne, P. E. S. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2007.05.001 · Cited by 806 · see citing works
The British Nationality Act as a logic program
Sergot, M. J., Sadri, F., Kowalski, R. A., Kriwaczek, F., Hammond, P., & Cory, H. T. (1986) · Communications of the ACM · doi:10.1145/5689.5920 · Cited by 564 · see citing works
A general approach for predicting the behavior of the Supreme Court of the United States
Katz, D. M., Bommarito, M. J., & Blackman, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0174698 · Cited by 562 · see citing works
Comparison of three one-question, post-task usability questionnaires
Sauro, J., & Dumas, J. S. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518946 · Cited by 511 · see citing works
Enriching Word Vectors with Subword InformationPreprint
Bojanowski, P., Grave, É., Joulin, A., & Mikolov, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04606 · Cited by 457 · see citing works
Lippi, M., & Torroni, P. (2016) · ACM Transactions on Internet Technology · doi:10.1145/2850417 · Cited by 379 · see citing works
Teaching case-based argumentation through a model and examples
Aleven, V. (1997) · Cited by 224 · see citing works
Automatically classifying case texts and predicting outcomes
Ashley, K. D., & Brüninghaus, S. (2009) · Artificial Intelligence and Law · doi:10.1007/s10506-009-9077-9 · Cited by 159 · see citing works
Predicting outcomes of case based legal arguments
Brüninghaus, S., & Ashley, K. D. (2003) · International Conference on Artificial Intelligence and Law · doi:10.1145/1047788.1047838 · Cited by 130 · see citing works
Predicting the Law Area and Decisions of French Supreme Court Cases
Şulea, O.-M., Zampieri, M., Vela, M., & van Genabith, J. (2017) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-049-6_092 · Cited by 114 · see citing works
BankXX: Supporting legal arguments through heuristic retrieval
Michener, E. R., Skalak, D. B., & Friedman, M. T. (1996) · Artificial Intelligence and Law · doi:10.1007/bf00123994 · Cited by 85 · see citing works
Recognizing cited facts and principles in legal judgements
Shulayeva, O., Siddharthan, A., & Wyner, A. (2017) · Artificial Intelligence and Law · doi:10.1007/s10506-017-9197-6 · Cited by 75 · see citing works
The Multiple Resources Questionnaire (MRQ)
Boles, D. B., & Adair, L. P. (2001) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120104502507 · Cited by 74 · see citing works
Evaluating a learning environment for case-based argumentation skills
Aleven, V., & Ashley, K. D. (1997) · International Conference on Artificial Intelligence and Law · doi:10.1145/261618.261650 · Cited by 66 · see citing works
Michener, E. R., Ashley, K., & Branting, L. K. (2005) · The Knowledge Engineering Review · doi:10.1017/s0269888906000701 · Cited by 59 · see citing works
Reasoning with Rules and Precedents - A Computational Model of Legal Analysis
Branting, L. K. (1999) · Medical Entomology and Zoology · Cited by 51 · see citing works
Using Machine Learning to Predict Outcomes in Tax LawPreprint
Alarie, B., Niblett, A., & Yoon, A. H. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2855977 · Cited by 46 · see citing works
Rules for a Flat World: Why Humans Invented Law and How to Reinvent It for a Complex Global Economy
Hadfield, G. K. (2016) · OUP Catalogue · Cited by 43 · see citing works
Reasoning with Rules and Precedents
Branting, L. K. (2000) · Springer eBooks · doi:10.1007/978-94-017-2848-5 · Cited by 43 · see citing works
Rule-based models of legal expertise
WATERMAN, D. A., & Peterson, M. A. (1980) · Cited by 41 · see citing works
Toward adding knowledge to learning algorithms for indexing legal cases
Brüninghaus, S., & Ashley, K. D. (1999) · International Conference on Artificial Intelligence and Law · doi:10.1145/323706.323709 · Cited by 33 · see citing works
Evaluation of semantic events for legal case retrieval
Maxwell, K. T., Oberlander, J., & Lavrenko, V. P. (2009) · doi:10.1145/1506250.1506259 · Cited by 19 · see citing works
Inducing Predictive Models for Decision Support in Administrative Adjudication
Branting, L. K., Yeh, A., Weiss, B., Merkhofer, E., & Brown, B. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00178-0_32 · Cited by 19 · see citing works
Self-citationRisk analysis for intellectual property litigation
Surdeanu, M., Nallapati, R. M., Gregory, G., Walker, J. H., & Manning, C. D. (2011) · International Conference on Artificial Intelligence and Law · doi:10.1145/2018358.2018375 · Cited by 17 · see citing works
Finding the right balance in artificial intelligence and law
McCarty, L. T. (2018) · Edward Elgar Publishing eBooks · doi:10.4337/9781786439055.00013 · Cited by 17 · see citing works
Supporting Legal Arguments through Heuristic Retrieval
Rissland, E. L., Skalak, D. B., & Friedman, M. (1994) · Cited by 11 · see citing works
Noise Induced Hearing Loss: An Application of the Angelic Methodology
Al-Abdulkarim, L., Atkinson, K. M., Bench-Capon, T. J. M., Whittle, S., Rob, W., & Wolfenden, C. (2017) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-838-9-79 · Cited by 6 · see citing works
Flagged first, then by citations
3 of the 35 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Explanation in AI and law: Past, present and future
Atkinson, K. M., Bench-Capon, T. J. M., & Bollegala, D. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103387 · Cited by 179 · see citing works
Explainable AI tools for legal reasoning about cases: A study on the European Court of Human Rights
Collenette, J., Atkinson, K. M., & Bench-Capon, T. J. M. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103861 · Cited by 81 · see citing works
Šavelka, J., & Ashley, K. D. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1279794 · Cited by 74 · see citing works
An end-to-end joint model for evidence information extraction from court record document
Ji, D., Tao, P., Fei, H., & Ren, Y. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102305 · Cited by 59 · see citing works
Suksi, M. (2021) · Artificial Intelligence and Law · doi:10.1007/s10506-020-09269-x · Cited by 55 · see citing works
Tagarelli, A., & Simeri, A. (2021) · Artificial Intelligence and Law · doi:10.1007/s10506-021-09301-8 · Cited by 55 · see citing works
Unlocking Practical Applications in Legal Domain
Šavelka, J. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595161 · Cited by 46 · see citing works
A Brief History of the Changing Roles of Case Prediction in AI and Law
Ashley, K. D. (2019) · Law in context · doi:10.26826/law-in-context.v36i1.88 · Cited by 31 · see citing works
Chen, H., Pieptea, L. F., & Ding, J. (2022) · IEEE Transactions on Reliability · doi:10.1109/tr.2022.3156126 · Cited by 31 · see citing works
Impact of e-court systems implementation: a case study
Ahmed, R. K., Muhammed, K. H., Pappel, I., & Draheim, D. (2021) · Transforming Government People Process and Policy · doi:10.1108/tg-01-2020-0008 · Cited by 29 · see citing works
Westermann, H., Šavelka, J., Walker, V. R., Ashley, K. D., & Benyekhlef, K. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200860 · Cited by 24 · see citing works
Šavelka, J., Westermann, H., Benyekhlef, K., Alexander, C. S., Grant, J. C., Amariles, D. R., Hamdani, R. E., Meeùs, S., Troussel, A., Araszkiewicz, M., et al. (2021) · International Conference on Artificial Intelligence and Law · doi:10.1145/3462757.3466149 · Cited by 22 · see citing works
Self-citationFrom Legal Contracts to Formal Specifications: A Systematic Literature Review
Soavi, M., Zeni, N., Mylopoulos, J., & Mich, L. (2022) · SN Computer Science · doi:10.1007/s42979-022-01228-4 · Cited by 22 · see citing works
An Explainable Approach to Deducing Outcomes in European Court of Human Rights Cases Using ADFs
Collenette, J., Atkinson, K. M., & Bench-Capon, T. J. M. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200488 · Cited by 22 · see citing works
Legal Knowledge and Information Systems
Saarbrücken, J. (. 2., Francesconi, E., Borges, G., & Sorge, C. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia362 · Cited by 16 · see citing works
Legal IR and NLP: The History, Challenges, and State-of-the-Art
Ganguly, D., Conrad, J. G., Ghosh, K., Ghosh, S., Goyal, P., Bhattacharya, P., Nigam, S. K., & Paul, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_34 · Cited by 17 · see citing works
An LLMs-based neuro-symbolic legal judgment prediction framework for civil cases
Wei, B., Yu, Y., Gan, L., & Wu, F. Q. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09433-1 · Cited by 15 · see citing works
MARRO: Multi-headed Attention for Rhetorical Role Labeling in Legal Documents
Bambroo, P., Adhikary, S., Bhattacharya, P., Chakraborty, A., Ghosh, S., & Ghosh, K. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09449-7 · Cited by 14 · see citing works
Self-citationSichelman, T. M. (2021) · Frontiers in Physics · doi:10.3389/fphy.2021.665054 · Cited by 13 · see citing works
Boniol, P., Panagopoulos, G., Xypolopoulos, C., Hamdani, R. E., Amariles, D. R., & Vazirgiannis, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06251 · Cited by 13 · see citing works
A Study on Legal Judgment Prediction using Deep Learning Techniques
Madambakam, P., & Rajmohan, S. (2022) · IEEE Silchar Subsection Conference (SILCON) · doi:10.1109/silcon55242.2022.10028879 · Cited by 10 · see citing works
Hybrid AI Framework for Legal Analysis of the EU Legislation Corrigenda
Palmirani, M., Sovrano, F., Liga, D., Sapienza, S., & Vitali, F. (2021) · Frontiers in artificial intelligence and applications · doi:10.3233/faia210319 · Cited by 10 · see citing works
Scalable Methods for Annotating Legal-Decision Corpora
Ferro, L., Aberdeen, J. S., Branting, K., Pfeifer, C., Yeh, A., & Chakraborty, A. (2019) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/w19-2202 · Cited by 8 · see citing works
Self-citationEffects of XAI on Legal Process
Nielsen, A., Skylaki, S., Norkutė, M., & Stremitzer, A. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595128 · Cited by 7 · see citing works
JURI SAYS: An Automatic Judgement Prediction System for the European Court of Human Rights
Medvedeva, M., Xu, X., Wieling, M. B., & Vols, M. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200883 · Cited by 8 · see citing works
Paley, A., Zhao, A. L. L., Pack, H., Servantez, S., Adler, R. F., Sterbentz, M., Pah, A. R., Schwartz, D. L., Barrie, C., Einarsson, A., et al. (2021) · International Conference on Artificial Intelligence and Law · doi:10.1145/3462757.3466100 · Cited by 6 · see citing works
Mozer, R., Miratrix, L. W., Relyea, J. E., & Kim, J. S. (2023) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986231207886 · Cited by 6 · see citing works
Information Extraction/Entailment of Common Law and Civil Code
Hudzina, J., Madan, K., Chinnappa, D., Harmouche, J., Bretz, H., Vold, A., & Schilder, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-79942-7_17 · Cited by 6 · see citing works
Analogical Reasoning, Generalization, and Rule Learning for Common Law Reasoning
Blass, J. A., & Forbus, K. D. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595121 · Cited by 4 · see citing works
AI in the Courtroom: Enhancing Legal Decision-Making through Predictive Modelling
Varshini, S. H., Varma, G. S., Pati, P. B., Prabhakar, P., & Parida, S. (2024) · Journal of Information & Knowledge Management · doi:10.1142/s0219649224500990 · Cited by 3 · see citing works
Digital Transformation Of The Judiciary
Fadeev, A. V. (2021) · The European Proceedings of Social & Behavioural Sciences · doi:10.15405/epsbs.2021.04.02.155 · Cited by 3 · see citing works
The Illinois Intentional Tort Qualitative Dataset
Blass, J. A., & Forbus, K. D. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220459 · Cited by 3 · see citing works
An Artificial Intelligence based Analysis in Legal domain
(2019) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.b1113.1292s219 · Cited by 3 · see citing works
Data Quality Evaluation and Improvement for Machine Learning
Chen, H. (2022) · doi:10.12794/metadc1944318 · Cited by 3 · see citing works
AsyLex: A Dataset for Legal Language Processing of Refugee Claims
Barale, C., Klaisoongnoen, M., Minervini, P., Rovatsos, M., & Bhuta, N. (2023) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2023.nllp-1.24 · Cited by 2 · see citing works
National Report on Automation in Decision-Making in Public Administration in Slovakia
Jakab, R. (2024) · AUC IURIDICA · doi:10.14712/23366478.2024.28 · Cited by 1 · see citing works
Wei, B. (2025) · Computer law & security review · doi:10.1016/j.clsr.2025.106166 · Cited by 1 · see citing works
Analysis on Hybrid Deep Neural Networks for Legal Domain Multitasks
Vaissnave, V., & Deepalakshmi, P. (2022) · International Journal of e-Collaboration · doi:10.4018/ijec.301257 · Cited by 1 · see citing works
Litigation Outcome Prediction, Access to Justice, and Legal Endogeneity
Alexander, C. S. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009255301.010 · Cited by 1 · see citing works
Automating the Classification of Finding Sentences for Linguistic Polarity.
Walker, V. R., Strong, S. R., & Walker, V. E. (2020)
Application of Hierarchical Clusters to Obtain Legal Reference Structures
Massa-Sánchez, P., Baño, N. F. P., Armijos, G., & Ordóñez, R. E. R. (2021) · Lecture notes in networks and systems · doi:10.1007/978-981-16-0739-4_6
Evaluating AI Models in Indian Legal Practice: A Study on Large Language Models
Sharma, K., Shukla, P., & Dwivedi, V. K. (2024) · IEEE India Conference · doi:10.1109/indicon63790.2024.10958512
Méthodes d'apprentissage automatique pour l'analyse de corpus jurisprudentiels
Condevaux, C. (2021) · theses.fr (ABES) · doi:10.70675/a07b8042z06e3z4d21z97fbz209b3ca1618e
- Authors
- Branting, K., Weiss, B., Brown, B., Pfeifer, C., Chakraborty, A., Ferro, L., Pfaff, M., & Yeh, A.
- Affiliations
- The MITRE Corporation, McLean, VA, USA
- Mitre (United States) (US)
- The MITRE Corporation, Ann Arbor, MI, USA
- Venue
- Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law
- Publisher
- ACM
- Published
- 2019-06-17
- Pages
- 22-31
- OpenAlex
- W2963594477
- Cited by
- 43
- References
- 35
- Citations by year
- 2025: 3 · 2024: 3 · 2023: 9 · 2022: 7 · 2021: 10 · 2020: 8 · 2019: 3
- FWCI
- 48.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Artificial Intelligence and Law (publishedVersion)
- Topics
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- legal decision support, semi-supervised learning, attentional networks, scale, case annotations, textual patterns, judicial decision prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review