The DOI matches its Crossref record.
ChatGPT Goes to Law School
Open accessAbstract
How well can AI models write law school exams without human assistance? To find out
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384
“For instance, when required to include up-to-date references, ChatGPT may fabricate ones that seem plausible but do not point to real-world sources (Choi et al., 2023).”
BackgroundChatGPT for Education and Research: Opportunities, Threats, and Strategies
Rahman, M. M., & Watanobe, Y. (2023) · Applied Sciences · doi:10.3390/app13095783 · Cited by 1,080
“[48] 66 Examine the performance for law school exams”
BackgroundRole of AI chatbots in education: systematic literature review
Labadze, L., Grigolia, M., & Machaidze, L. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00426-1 · Cited by 1,043
“Students can also use ChatGPT to quiz themselves on various subjects, reinforcing their knowledge and preparing for exams (Choi et al., 2023; Eysenbach, 2023; Sevgi et al., 2023; Thurzo et al., 2023).”
BackgroundRevolutionizing education with AI: Exploring the transformative potential of ChatGPT
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13152 · Cited by 975
“The innate abilities of ChatGPT have been showcased in studies that indicate its success in passing a Law School examination (Choi et al., 2023) and a Master of Business Administration (MBA) examination (Terwiesch, 2023).”
BackgroundBoscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005439 · Cited by 620
ChatGPT and the rise of generative AI: Threat to academic integrity?
Eke, D. O. (2023) · Journal of Responsible Technology · doi:10.1016/j.jrt.2023.100060 · Cited by 532
Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2024) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2023.0254 · Cited by 420
Rao, A., Pang, M., Kim, J., Kamineni, M., Lie, W., Prasad, A. K., Landman, A., Dreyer, K., & Succi, M. D. (2023) · Journal of Medical Internet Research · doi:10.2196/48659 · Cited by 412
Lee, D., Arnold, M., Srivastava, A., Plastow, K., Strelan, P., Ploeckl, F., Lekkas, D., & Palmer, E. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100221 · Cited by 402
Welcome to the Era of ChatGPT et al.: The Prospects of Large Language Models
Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W., & Hinz, O. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00795-x · Cited by 384
AI literacy and its implications for prompt engineering strategies
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100225 · Cited by 368
Gender bias and stereotypes in Large Language Models
Kotek, H., Dockum, R., & Sun, D. (2023) · Proceedings of The ACM Collective Intelligence Conference · doi:10.1145/3582269.3615599 · Cited by 381
Ali, R., Tang, O. Y., Connolly, I. D., Fridley, J. S., Shin, J. H., Zadnik Sullivan, P. L., Cielo, D., Oyelese, A. A., Doberstein, C. E., Telfeian, A. E., et al. (2023) · Neurosurgery · doi:10.1227/neu.0000000000002551 · Cited by 354
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335
Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S., Lyden, S., Neal, P., & Sandison, C. (2023) · European Journal of Engineering Education · doi:10.1080/03043797.2023.2213169 · Cited by 336
Is ChatGPT Leading Generative AI? What is Beyond Expectations?
AYDIN, Ö., & KARAARSLAN, E. (2023) · Academic Platform Journal of Engineering and Smart Systems · doi:10.21541/apjess.1293702 · Cited by 311
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024) · Journal of Legal Analysis · doi:10.1093/jla/laae003 · Cited by 559
ChatGPT and Software Testing Education: Promises & Perils
Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023) · 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw58534.2023.00078 · Cited by 270
“Our results are in contrast to the results of applying ChatGPT in other domains, such as in medicine [26] or law [27] where ChatGPT is shown to pass certain parts of their exams.”
BackgroundResultA comprehensive survey of ChatGPT: Advancements, applications, prospects, and challenges
Nazir, A., & Wang, Z. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100022 · Cited by 245
Exploring the integration of ChatGPT in education: adapting for the future
Elbanna, S., & Armstrong, L. (2024) · Management & Sustainability: An Arab Review · doi:10.1108/msar-03-2023-0016 · Cited by 238
Lai, V. D., Ngo, N. T., Veyseh, A. P. B., Man, H., Dernoncourt, F., Bui, T., & Nguyen, T. H. (2023) · arXiv · doi:10.48550/arxiv.2304.05613 · Cited by 233
Lai, V., Ngo, N., Pouran Ben Veyseh, A., Man, H., Dernoncourt, F., Bui, T., & Nguyen, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.878 · Cited by 233
Farazouli, A., Cerratto-Pargman, T., Bolander-Laksov, K., & McGrath, C. (2024) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2023.2241676 · Cited by 219
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211
“Lawyer Qualification Test: Including the bar exam, these tests assess legal knowledge, analytical skills, and ethical understanding.”
Oh, N., Choi, G.-S., & Lee, W. Y. (2023) · Annals of Surgical Treatment and Research · doi:10.4174/astr.2023.104.5.269 · Cited by 201
“Notably, the system achieved high accuracy on the United States Medical Licensing Examination, the Bar exam, and the Wharton MBA final exam, even without fine-tuning the pretrained model [2-5].”
Ansari, A. N., Ahmad, S., & Bhutta, S. M. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12223-4 · Cited by 198
Role of activity-based learning and ChatGPT on students' performance in education
Al Shloul, T., Mazhar, T., Abbas, Q., Iqbal, M., Ghadi, Y. Y., Shahzad, T., Mallek, F., & Hamam, H. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100219 · Cited by 173
“This multidisciplinary support is crucial in educational settings where the ability to tailor learning experiences to individual needs is increasingly valued (Choi et al., 2023).”
BackgroundLet's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
Shahriar, S., & Hayawi, K. (2023) · arXiv · doi:10.48550/arxiv.2302.13817 · Cited by 167
Let’s Have a Chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
Shahriar, S., & Hayawi, K. (2023) · Artificial Intelligence and Applications · doi:10.47852/bonviewaia3202939 · Cited by 167
Wood, D. A., Achhpilia, M. P., Adams, M. T., Aghazadeh, S., Akinyele, K., Akpan, M., Allee, K. D., Allen, A. M., Almer, E. D., Ames, D., et al. (2023) · Issues in Accounting Education · doi:10.2308/issues-2023-013 · Cited by 166
ChatGPT for Education and Research: Opportunities, Threats, and Strategies
Rahman, M. M., & Watanobe, Y. (2023) · doi:10.20944/preprints202303.0473.v1 · Cited by 1,080
Leveraging ChatGPT to Aid Construction Hazard Recognition and Support Safety Education and Training
Uddin, S. M. J., Albert, A., Ovid, A., & Alsharef, A. (2023) · Sustainability · doi:10.3390/su15097121 · Cited by 158
Ibrahim, H., Liu, F., Asim, R., Battu, B., Benabderrahmane, S., Alhafni, B., Adnan, W., Alhanai, T., AlShebli, B., Baghdadi, R., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-38964-3 · Cited by 154
Artificial intelligence for teaching and learning in schools: The need for pedagogical intelligence
Díaz, B., & Nussbaum, M. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105071 · Cited by 148
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137
“…built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymakers (J & C, 2023; Bartz, 2023; Lieu, 2023), and scholars from many fields (Chan, 2022; Lund & Wang, 2023; Choi et al., 2023; Biswas, 2023).”
Assessing the utility of ChatGPT throughout the entire clinical workflow
Rao, A., Pang, M., Kim, J., Kamineni, M., Lie, W., Prasad, A. K., Landman, A., Dreyer, K. J., & Succi, M. D. (2023) · doi:10.1101/2023.02.21.23285886 · Cited by 134
The Educational Affordances and Challenges of ChatGPT: State of the Field
Crompton, H., & Burke, D. (2024) · TechTrends · doi:10.1007/s11528-024-00939-0 · Cited by 141
“In addition, it can be used in educational disciplines such as medicine (Sed-aghat, 2023), law (Choi et al., 2023), journalism (Pavlik, 2023), and engineering (Qadir, 2022).”
Exploring ChatGPT for information of cardiopulmonary resuscitation
Ahn, C. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2023.109729 · Cited by 112
Redefining entrepreneurship education in the age of artificial intelligence: An explorative analysis
Vecchiarini, M., & Somià, T. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100879 · Cited by 124
Faith and Fate: Limits of Transformers on Compositionality
Dziri, N., Lu, X., Sclar, M., Li, X. (., Jiang, L., Lin, B. Y., Welleck, S., West, P., Bhagavatula, C., Bras, R. L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3081 · Cited by 335
“” – Charles Dickens, A Tale of Two Cities Large-scale Transformers such as ChatGPT [40] and GPT-4 [41] demonstrate unprecedented capabilities [40, 56, 8, 11, 65], even noted as “sparks of AGI” [9].”
BackgroundFaith and Fate: Limits of Transformers on Compositionality
Dziri, N., Lu, X., Sclar, M., Li, X. L., Jiang, L., Lin, B. Y., West, P., Bhagavatula, C., Bras, R. L., Hwang, J. D., et al. (2023) · arXiv · Cited by 124
Moshirfar, M., Altaf, A. W., Stoakes, I. M., Tuttle, J. J., & Hoopes, P. C. (2023) · Cureus · doi:10.7759/cureus.40822 · Cited by 124
Lai, C. Y., Cheung, K. Y., Chan, C. S., & Law, K. K. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100246 · Cited by 122
ChatGPT has Aced the Test of Understanding in College Economics: Now What?
Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023) · The American Economist · doi:10.1177/05694345231169654 · Cited by 119
“Several studies have demonstrated ChatGPT ’ s ability to pass standardized tests in various fi elds such as mathematics (Leswing, 2023), medicine (Gilson et al., 2023), law (Choi et al., 2023), and physics (West, 2023).”
BackgroundPerformance of ChatGPT on the pharmacist licensing examination in Taiwan
Wang, Y.-M., Shen, H.-W., & Chen, T.-J. (2023) · Journal of the Chinese Medical Association · doi:10.1097/jcma.0000000000000942 · Cited by 109
Can large language models write reflectively
Li, Y., Sha, L., Yan, L., Lin, J., Raković, M., Galbraith, K., Lyons, K., Gašević, D., & Chen, G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100140 · Cited by 99
Pardos, Z. A., & Bhandari, S. (2024) · PLOS ONE · doi:10.1371/journal.pone.0304013 · Cited by 97
“This corroborates past observations that ChatGPT 3.5 scores at the C+ level on the bar exam [66] and below passing on medical exams [67].”
ResultInfluentialGraduate Instructors Navigating the AI Frontier: The Role of ChatGPT in Higher Education
Parker, L., Carter, C., Karakas, A., Loper, A. J., & Sokkar, A. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100166 · Cited by 95
Spennemann, D. H. R. (2023) · Knowledge · doi:10.3390/knowledge3030032 · Cited by 93
Assessing the Benefits of ChatGPT for Business: An Empirical Study on Organizational Performance
Chu, M.-N. (2023) · IEEE Access · doi:10.1109/access.2023.3297447 · Cited by 69
Seeing ChatGPT Through Students' Eyes: An Analysis of TikTok Data
Haensch, A.-C., Ball, S., Herklotz, M., & Kreuter, F. (2023) · arXiv · doi:10.48550/arxiv.2303.05349 · Cited by 62
ChatGPT and other AI Users: Innovative and Creative Utilitarian Value and Mindset Shift
Duhaylungsod, A. V., & Chavez, J. V. (2023) · Journal of Namibian Studies : History Politics Culture · doi:10.59670/jns.v33i.2791 · Cited by 59
Malik, A., Khan, M. L., Hussain, K., Qadir, J., & Tarhini, A. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2024.2409407 · Cited by 57
Navigating Generative AI (ChatGPT) in Higher Education: Opportunities and Challenges
Wang, T. (2023) · Lecture Notes in Educational Technology · doi:10.1007/978-981-99-5961-7_28 · Cited by 56
Maurya, R. K. (2024) · Counselling and Psychotherapy Research · doi:10.1002/capr.12699 · Cited by 51
Hu, J.-M., Liu, F.-C., Chu, C.-M., & Chang, Y.-T. (2023) · Journal of Medical Internet Research · doi:10.2196/49385 · Cited by 50
Cheng, M. W. T., & YIM, I. H. Y. (2024) · Discover Education · doi:10.1007/s44217-023-00081-8 · Cited by 44
Sharma, P., Thapa, K., Thapa, D., Dhakal, P., Upadhaya, M. D., Adhikari, S., & Khanal, S. R. (2023) · arXiv · doi:10.48550/arxiv.2307.00112 · Cited by 40
Generative AI and the future of connectivist learning in higher education
Liang, E. S., & Bai, S. (2025) · Journal of Asian Public Policy · doi:10.1080/17516234.2024.2386085 · Cited by 34
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
(2024) · Proceedings of the 30th Conference on Pattern Languages of Programs · doi:10.64346/plop2023p05 · Cited by 32
“The use of ChatGPT in Law School has also been explored [34].”
MethodIncorporating AI tools into medical education: harnessing the benefits of ChatGPT and Dall-E
Miftahul Amri, M., & Khairatun Hisan, U. (2023) · Journal of Novel Engineering Science and Technology · doi:10.56741/jnest.v2i02.315 · Cited by 31
Judge Juan Manuel Padilla Garcia, ChatGPT, and a controversial medicolegal milestone
Parikh, P. M., Shah, D. M., & Parikh, K. P. (2023) · Indian Journal of Medical Sciences · doi:10.25259/ijms_31_2023 · Cited by 24
Public attitudes toward chatgpt on twitter: sentiments, topics, and occupations
Koonchanok, R., Pan, Y., & Jang, H. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01260-7 · Cited by 25
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2023) · arXiv · doi:10.48550/arxiv.2307.12267 · Cited by 25
Danish university policies on generative AI: Problems, assumptions and sustainability blind spots
Driessens, O., & Pischetola, M. (2024) · MedieKultur: Journal of media and communication research · doi:10.7146/mk.v40i76.143595 · Cited by 25
Luke, W. A. N. V., Seow Chong, L., Ban, K. H., Wong, A. H., Zhi Xiong, C., Shuh Shing, L., Taneja, R., Samarasekera, D. D., & Yap, C. T. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2308779 · Cited by 24
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
Balloccu, S., Schmidtová, P., Lango, M., & Dušek, O. (2024) · arXiv · doi:10.48550/arxiv.2402.03927 · Cited by 20
Cao, X., Lin, Y.-J., Zhang, J.-H., Tang, Y.-P., Zhang, M.-P., & Gao, H.-Y. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13339-5 · Cited by 22
“Some studies have found that ChatGPT has ‘sat’ examinations it has often performed well, scoring around the average grade, especially in writing English compositions (Choi et al., 2021).”
Chatbots: A Framework for Improving Information Security Behaviours using ChatGPT
Gundu, T. (2023) · IFIP Advances in Information and Communication Technology · doi:10.1007/978-3-031-38530-8_33 · Cited by 14
Quantum leap in medical mentorship: exploring ChatGPT’s transition from textbooks to terabytes
Chokkakula, S., Chong, S., Yang, B., Jiang, H., Yu, J., Han, R., Attitalla, I. H., Yin, C., & Zhang, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1517981 · Cited by 12
ChatGPT in Research and Education: A SWOT Analysis of Its Academic Impact
Miah, A. S. M., Tusher, M. M. R., Hossain, M. M., Hossain, M. M., Rahim, M. A., Hamid, M. E., Islam, M. S., & Shin, J. (2025) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2025.064168 · Cited by 11
Jan, R. (2024) · Advances in Educational Technologies and Instructional Design · doi:10.4018/979-8-3693-0884-4.ch014 · Cited by 9
The Cambridge Law Corpus: A Dataset for Legal AI Research
Östling, A., Sargeant, H., Xie, H., Bull, L., Terenin, A., Jonsson, L., Magnusson, M., & Steffek, F. (2024) · doi:10.2139/ssrn.4763429 · Cited by 8
Public Attitudes Toward ChatGPT on Twitter: Sentiments, Topics, and Occupations
Koonchanok, R., Pan, Y., & Jang, H. (2023) · arXiv · doi:10.48550/arxiv.2306.12951 · Cited by 5
ChatGPT and the Future of Writing about Writing
De Matas, J. (2023) · Double Helix: A Journal of Critical Thinking and Writing · doi:10.37514/dbh-j.2023.11.1.09 · Cited by 4
Untitled work
Ovcharov, V. (2026) · arXiv
“Large language models (LLMs) have achieved remarkable performance on legal reasoning tasks, passing bar examinations [11] and generating persuasive legal memoranda [4].”
Metadata
- Authors
- Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B.
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2023
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W4320009668
- Cited by
- 485
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.