CoChat
Sign Up
Cited by 16Artificial Intelligence in LawSoftware Engineering ResearchAI-based Problem Solving and Planning

Automation of Legal Precedents Retrieval: Findings from a Literature Review

Open access
Mentzingen, Hugo, António, Nuno, Bacao, Fernando·2023·International Journal of Intelligent Systems
DOI 10.1155/2023/6660983
Open this paper in CoChat — free accountOpen PDF

Abstract

Judges frequently rely their reasoning on precedents. Courts must preserve uniformity in decisions while, depending on the legal system, previous cases compel rulings. The search for methods to accurately identify similar previous cases is not new and has been a vital input, for example, to case‐based reasoning (CBR) methodologies. This literature review offers a comprehensive analysis of the advancements in automating the identification of legal precedents, primarily focusing on the paradigm shift from manual knowledge engineering to the incorporation of Artificial Intelligence (AI) technologies such as natural language processing (NLP) and machine learning (ML). While multiple approaches harnessing NLP and ML show promise, none has emerged as definitively superior, and further validation through statistically significant samples and expert‐provided ground truth is imperative. Additionally, this review employs text‐mining techniques to streamline the survey process, providing an accurate and holistic view of the current research landscape. By delineating extant research gaps and suggesting avenues for future exploration, this review serves as both a summation and a call for more targeted, empirical investigations.

Integrity signals

Registry

Crossref · 10.1155/2023/6660983

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Our typology of legal RAG errors suggests methodological paths for improving systems, such as query expansion prior to retrieval, improved representation of case metadata, deepening interaction between the retriever and the generator through joint training and with domain- or task-adaptation.

    “our typology of legal RAG errors suggests methodological paths for improving systems, such as query expansion prior to retrieval (Carpineto and Romano  2012), improved representation of case metadata (e.g., precedents (Mentzingen et al. 2023), holdings (Zheng et al. 2021), and entities (Leitner et al. 2019)), deepening interaction between the retriever and the generator through joint training”

    From Hallucination‐Free? Assessing the Reliability of Leading <scp>AI</scp> Legal Research Tools (2025) · cites this paper as Mentzingen et al. 2023

Cites

Flagged first, then by citations

Research works printed but not linked: 96

  1. Gomez A. R., 2021, Demand side justice, Georgetown Journal on Poverty Law and Policy, 411
  2. Rhode D. L., 2004, Access to Justice
  3. Popova O., 2021, The Undeniable Benefits of Court Automation
  4. Susskind R., 2020, The future of courts, Practice, 6
  5. 10.1093/jnlids/idq025 · DOI
  6. 10.1017/s1352325214000044 · DOI
  7. 10.1016/j.irle.2007.01.005 · DOI
  8. 10.1007/bf00155578 · DOI
  9. RothA. Case-based reasoning in the law: a formal theory of reasoning by case comparison 2003 Universiteit Maastricht Maastricht Netherlands Doctoral Thesis.
  10. 10.1007/978-3-031-16474-3_5 · DOI
  11. MentzingenH. AntónioN. andBacaoF. Automation of legal precedents retrieval: findings from a rapid literature review 2022 https://www.researchsquare.com/article/rs-2292464/v1. · DOI
  12. 10.1016/j.jclinepi.2009.06.005 · DOI · Paper page
  13. 10.7759/cureus.864 · DOI
  14. 10.1002/14651858.ed000048 · DOI
  15. 10.1186/1748-5908-5-56 · DOI · Paper page
  16. 10.1186/2046-4053-1-10 · DOI · Paper page
  17. Tricco A. C., 2017, Rapid Reviews to Strengthen Health Policy and Systems: A Practical Guide
  18. FengL. ChiamY. K. andLoS. K. Text-mining techniques and tools for systematic literature reviews: a systematic literature review Proceedings of the Asia-Pacific Software Engineering Conference APSEC December 2018 Nara Japan 41–50. · DOI
  19. 10.18089/tms.2019.15011 · DOI
  20. SchweizerV. J. KurniawanJ. H. andPowerA. Semi-automated literature review for scientific assessment of socioeconomic climate change scenarios Proceedings of the Companion Proceedings of the Web Conference 2022 August 2022 Lyon France 789–799. · DOI
  21. 10.3390/info13030117 · DOI
  22. Van Rossum G., 2009, Python 3 Reference Manual
  23. 10.1007/s10506-020-09280-2 · DOI
  24. 10.1007/s11192-015-1765-5 · DOI · Paper page
  25. 10.1186/1751-0473-6-7 · DOI
  26. 10.1186/s40537-019-0255-7 · DOI
  27. 10.1162/jmlr.2003.3.4-5.993 · DOI
  28. Rehurek R., 2011, Gensim-python Framework for Vector Space Modelling
  29. Hu D. J., 2009, Latent Dirichlet Allocation for Text, Images, and Music
  30. AroraJ. PatankarT. ShahA. andJoshiS. Artificial intelligence as legal research assistant Proceedings of the CEUR Workshop Proceedings 2826 December 2020 Hyderabad India 60–65.
  31. Di NunzioG. M. G. M. A study on lemma vs stem for legal information retrieval using R tidyverse. IMS UniPD @ AILA 2020 Task 1 Proceedings of the CEUR Workshop Proceedings 2826 December 2020 Hyderabad India 54–59.
  32. MoreR. PatilJ. PalaskarA. andPawdeA. Removing named entities to find precedent legal cases Proceedings of the CEUR Workshop Proceedings 2517 December 2019 Kolkata India 13–18.
  33. AminK. KapetanakisS. AlthoffK. D. DengelA. andPetridisM. Cases without borders: automating knowledge acquisition approach using deep autoencoders and siamese networks in case-based reasoning Proceedings of the International Conference on Tools with Artificial Intelligence ICTAI November 2019 Portland OR USA 133–140 https://doi.org/10.1109/ICTAI.2019.00027. · DOI
  34. BhattacharyaP. GhoshK. GhoshS. PalA. MehtaP. BhattacharyaA. andMajumderP. FIRE 2019 AILA track: artificial intelligence for legal assistance Proceedings of the ACM International Conference Proceeding Series 2517 February 2018 New York NY USA 4–6.
  35. Nair A. M. A. M., 2018, Similarity analysis of court judgements using association rule mining on case citation data-a case study, International Journal of Engineering Research and Technology, 11, 373
  36. KulkarniY. H. Y. H. PatilR. andShridharanS. Detection of catchphrases and precedence in legal documents Proceedings of the CEUR Workshop Proceedings 2036 December 2017 Kolkata India 86–89.
  37. ThumaE.andMotlogelwaN. P. N. P. On the importance of legal catchphrases in precedence retrieval Proceedings of the CEUR Workshop Proceedings 2036 December 2017 Kolkata India 92–94.
  38. OconitrilloL. R. R.andDe La Ossa OseguedaA. Business intelligence model to support a judge’s decision making about legal situations Proceedings of the 2016 IEEE 36th Central American and Panama Convention CONCAPAN 2016 November 2017 San Jose Costa Rica 1–5. · DOI
  39. 10.1109/access.2017.2745208 · DOI
  40. EyorokonV. B. BengfortB. PanjalaU. S. andCoxM. T. Goal trajectories for knowledge investigations Proceedings of the CEUR Workshop Proceedings 1815 December 2016 Atlanta GA USA 202–211.
  41. McLarenB. M.andAshleyK. D. Can temporal representation and reasoning make a difference in automated legal reasoning? Lessons from an AI-based ethical reasoner Proceedings of the International Conference on Artificial Intelligence and Law June 2011 Pittsburgh PA USA 229–238 https://doi.org/10.1145/2018358.2018391 2-s2.0-80052415574. · DOI
  42. RamanV.andPalanissamyA. SandhuP. S.andXieY. Computer aided legal support systems: methodology to automatically convert the legal text into legal cases for building case base reasoning Proceedings of the 2009 International Conference On Computer Engineering And Applications Issue International Conference on Computer Engineering and Applications January 2009 Washington DC USA 283–287.
  43. RamanV.andPalanissamyA. Computer aided legal support system: an initial framework retrieving legal cases by case base reasoning approach Proceedings of the 2008 International Conference on Innovations in Information Technology IIT 2008 December 2008 Al Ain United Arab Emirates 317–321 https://doi.org/10.1109/INNOVATIONS.2008.4781663 2-s2.0-67649479617. · DOI
  44. Nouaouria N., 2006, A case based tool as intelligent assistance to Mufti, Arabian Journal for Science and Engineering, 31, 75
  45. WangR.andZengY. Nonlinear nearest-neighbour matching and its application in legal precedent retrieval Proceedings of the 3rd International Conference on Information Technology and Applications ICITA 2005 July 2005 Sydney Australia 341–346 https://doi.org/10.1109/icita.2005.191. · DOI
  46. Liu C.-L., 2004, Case instance generation and refinement for case-based criminal summary judgments in Chinese, Journal of Information Science and Engineering, 20, 783
  47. 10.1016/s0004-3702(03)00135-8 · DOI
  48. 10.1080/088395101753199588 · DOI
  49. 10.1016/s0957-4174(99)00063-9 · DOI
  50. MikolovT. ChenK. CorradoG. andDeanJ. Distributed representations of words and phrases and their compositionality 2 NIPS’13: Proceedings of the 26th International Conference on Neural Information Processing Systems December 2013 Lake Tahoe NV USA 3111–3119.
  51. ChoK. Van MerriënboerB. GulcehreC. BahdanauD. BougaresF. SchwenkH. andBengioY. Learning phrase representations using RNN encoder-decoder for statistical machine translation Proceedings of the Conference EMNLP 2014 Conference on Empirical Methods in Natural Language Processing October 2014 Doha Qatar 1724–1734. · DOI · Paper page
  52. 10.1109/access.2018.2885698 · DOI
  53. 10.48550/arxiv.1706.03762 · DOI · Paper page
  54. HowardJ.andRuderS. Universal language model fine-tuning for text classification 1 Proceedings of the ACL 2018 56th Annual Meeting of the Association for Computational Linguistics July 2018 Melbourne Australia https://doi.org/10.18653/v1/p18-1031. · DOI · Paper page
  55. 10.1007/s11431-020-1647-3 · DOI · Paper page
  56. 10.1207/s15516709cog1902_1 · DOI
  57. 10.3233/978-1-58603-952-3-63 · DOI
  58. LeQ.andMikolovT. Distributed representations of sentences and documents 4 . Proceedings of the 31st International Conference on Machine Learning ICML 2014 June 2014 Beijing China 2931–2939.
  59. BaldiP. Autoencoders unsupervised learning and deep architectures Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning workshop July 2012 Washington DC USA 37–50 https://doi.org/10.1561/2200000006 2-s2.0-69349090197. · DOI
  60. Mansouri A., 2008, Named entity recognition approaches, Journal of Computer Science, 8, 339
  61. 10.1561/1500000019 · DOI · Paper page
  62. AngelovD. Top2vec: distributed representations of topics 2020 1–25 http://arxiv.org/abs/2008.09470. · arXiv
  63. 10.1145/582415.582416 · DOI
  64. Bird S., 2009, Natural Language Processing with Python
  65. Theodoridis S., 2020, Machine Learning: A Bayesian and Optimization Perspective
  66. 10.1147/rd.14.0309 · DOI
  67. 10.1108/eb026526 · DOI
  68. 10.1016/j.eswa.2015.02.055 · DOI
  69. AroraS. GeR. andMoitraA. Learning topic models- going beyond SVD Proceedings of the Annual IEEE Symposium on Foundations of Computer Science FOCS October 2012 New Brunswick NJ USA 1–10 https://doi.org/10.48550/arxiv.1204.1956. · DOI
  70. Calderon E. B. P., 2021, Case-based legal reasoning and inductive reasoning: determination of the input parameters using neutrosophic AHP, Neutrosophic Sets and Systems, 44, 71
  71. 10.1007/s10506-021-09283-7 · DOI
  72. 10.18201/ijisae.2021.238 · DOI
  73. AtkinsonK. CollenetteJ. Bench-CaponT. andDzehtsiarouK. Practical tools from formal models: the ECHR as a case study Proceedings of the 18th International Conference on Artificial Intelligence and Law ICAIL 2021 June 2021 São Paulo Brazil 170–174 https://doi.org/10.1145/3462757.3466095. · DOI
  74. Lefoane M., 2019, Legal statutes retrieval: a comparative approach on performance of title and statutes descriptive text, CEUR Workshop Proceedings, 2517, 52
  75. MorrisJ. User-friendly open-source case-based legal reasoning Proceedings of the 17th International Conference on Artificial Intelligence and Law ICAIL 2019 June 2019 Montreal Canada 270–271 https://doi.org/10.1145/3322640.3326717 2-s2.0-85071273204. · DOI
  76. YuanZ. Intelligent decision support system development technology of automotive mechanical system user expert admin man-machine interactive interface control and management module case library save case part feature information extraction case match success case Proceedings of the 3rd International Conference on Education Management and Computing Technology (ICEMCT 2016) June 2016 Hangzhou China 1373–1377. · DOI
  77. 10.1007/978-3-662-45960-7 · DOI
  78. NagataA. KobayashiT. HasegawaK. MorogaK. KuriyamaY. andOhnoT. Development of the case-based reasoning system for regional science and technology policy: an interim report Proceedings of the PICMET 2014- Portland International Center for Management of Engineering and Technology Proceedings: Infrastructure and Service Integration July 2014 Kanazawa Japan 135–139.
  79. 10.1016/j.sbspro.2014.03.705 · DOI
  80. 10.1016/j.autcon.2012.10.014 · DOI
  81. WynerA.andPetersW. CalzolariN. ChoukriK. MaegaardB. MarianiJ. OdijkJ. PiperidisS. RosnerM. andTapiasD. Towards annotating and extracting textual legal case factors Proceedings of The Lrec 2010 Seventh International Conference On Language Resources And Evaluation May 2010 Valletta Malta I36–I45.
  82. 10.1080/10286600902781633 · DOI
  83. 10.1007/978-3-642-12837-0_4 · DOI
  84. 10.1007/11590316_4 · DOI
  85. AshleyK. D. Case-based reasoning research and development 2689 Proceedings of the 5th International Conference on Case-Based Reasoning ICCBR 2003 June 2003 Trondheim Norway. · DOI
  86. HoeschlH. C. BuenoT. C. D. BortolonA. MattosE. S. RibeiroM. S. BarciaR. M. andTheissI. Structured contextual search for the un security council 2 ICEIS 2003 Proceedings of the 5th International Conference on Enterprise Information Systems April 2003 Angers France 100–107.
  87. 10.1016/s0004-3702(03)00109-7 · DOI
  88. 10.1023/a:1025450828280 · DOI
  89. 10.1007/978-3-319-74727-9_18 · DOI
  90. 10.1007/s00146-015-0596-5 · DOI
  91. Elhalwany I., 2015, Using textual case-based Reasoning in intelligent Fatawa QA system, The International Arab Journal of Information Technology, 12, 503
  92. Carneiro D., 2009, The legal precedent in online dispute resolution, Frontiers in Artificial Intelligence and Applications, 205
  93. 10.1016/s0004-3702(03)00102-4 · DOI
  94. LiuC.-L. C. L. ChangC. T. C.-T. andHoJ.-H. J. H. Classification and clustering for case-based criminal summary judgments Proceedings of the International Conference on Artificial Intelligence and Law June 2003 São Paulo Brazil 252–261 https://doi.org/10.1145/1047788.1047840 2-s2.0-3142624521. · DOI
  95. 10.1142/9789812794604_0010 · DOI
  96. HirotaK. TakamaY. XuM. andYoshinoH. Legal argument in fuzzy legal expert system (FLES) 1 Proceedings of the 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics Control and Instrumentation. 21st Century Technologies October 2000 Nagoya Japan 207–210 https://doi.org/10.1109/IECON.2000.973151 2-s2.0-84969135593. · DOI

Cited by

Sorted by citations

Authors
Mentzingen, H., António, N., & Bacao, F.
Affiliations
Universidade Nova de Lisboa (PT)
Corresponding author
Mentzingen, Hugo
ORCID
Venue
International Journal of Intelligent Systems
Publisher
Wiley
Published
2023-01
Volume
2023
Issue
1
ISSN
  • 0884-8173 (ISSN-L)
  • 1098-111X
DOI
10.1155/2023/6660983
OpenAlex
W4388206012
Cited by
16
References
5
Citations by year
2026: 6 · 2025: 4 · 2024: 6
FWCI
15.68
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Intelligent Systems (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
Funders
Fundação para a Ciência e a Tecnologia (UIDB/04152/2020)
Topics
  • Social Sciences › Law › Artificial Intelligence in Law
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
case-based reasoning, judicial decision-making, legal text mining, knowledge engineering, legal research automation, research gaps

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review