An analysis of three chatbots: BlenderBot, ChatGPT and LaMDA
Open accessAbstract
Summary: Google, Facebook, OpenAI, and others have released access to versions of language chatbots that they have developed. These chatbots have been trained on massive amounts of text using neural networks for language processing. Using an approach similar to security penetration testing, this paper investigates and compares three different chatbots, assessing potential strengths and limitations of these systems. The paper presents several findings, including a comparison of those systems across answers to common questions, an analysis of the use of names and activities to guide discussion in two systems, an analysis of the extent of differences in responses arising from “regeneration” of a question, the determination of a weakness in a system of knowing “who” invented something, development of a potential new subfield, sensitive topic classifiers, and an analysis of some of the implications of these findings. As part of this analysis, I find emerging topics in chatbots, such as “topic stalemate” and the use of sensitive topic classifiers.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI-powered Chatbots (e.g., BlenderBot and LaMDA) and AI text generators appear in the dataset with lower frequencies.
“AI-powered Chatbots, such as BlenderBot and LaMDA, and AI text generators (AITG) appear in the dataset, albeit with lower frequencies [73,110].”
From Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis (2023) · cites this paper as [110]
In a comparison by O'Leary, ChatGPT outperformed LaMDA and BlenderBot.
“For example, O’Leary [53] compared two chatbots, LaMDA and BlenderBot, with ChatGPT and revealed that ChatGPT outperformed both.”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [53]
Cites
Geerts, G. L., & McCarthy, W. E. (2002) · International Journal of Accounting Information Systems · doi:10.1016/s1467-0895(01)00020-3 · Cited by 295 · see citing works
Technical guide to information security testing and assessment.
Scarfone, K. A., Souppaya, M., Cody, A., & Orebaugh, A. (2008) · doi:10.6028/nist.sp.800-115 · Cited by 289 · see citing works
"On the internet, nobody knows you're a dog": a twitter case study of anonymity in social networks
Peddinti, S. T., Ross, K. W., & Cappos, J. (2014) · Proceedings of the second ACM conference on Online social networks · doi:10.1145/2660460.2660467 · Cited by 128 · see citing works
GOOGLE'S Duplex: Pretending to be human*
O’Leary, D. E. (2019) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1443 · Cited by 57 · see citing works
Self-citationA methodology for penetration testing
Pfleeger, C. P., Pfleeger, S. L., & Theofanos, M. F. (1989) · Computers & Security · doi:10.1016/0167-4048(89)90054-0 · Cited by 54 · see citing works
Armchair Auditors: Crowdsourcing Analysis of Government Expenditures
O’Leary, D. E. (2015) · Journal of Emerging Technologies in Accounting · doi:10.2308/jeta-51225 · Cited by 38 · see citing works
Self-citationMassive data language models and conversational artificial intelligence: Emerging issues
O’Leary, D. E. (2022) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1522 · Cited by 30 · see citing works
Self-citationVerification and validation of intelligent systems: Five years of aaai workshops
O’Leary, D. E. (1994) · International Journal of Intelligent Systems · doi:10.1002/int.4550090802 · Cited by 6 · see citing works
Self-citationO’Leary, D. E. (2008) · Wiley Encyclopedia of Computer Science and Engineering · doi:10.1002/9780470050118.ecse146 · Cited by 3 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023) · Sustainability · doi:10.3390/su151712983 · Cited by 784 · see citing works
ChatGPT in education: Methods, potentials, and limitations
Memarian, B., & Doleck, T. (2023) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2023.100022 · Cited by 266 · see citing works
A systematic literature review of empirical research on ChatGPT in education
Albadarin, Y., Saqr, M., Pope, N., & Tukiainen, M. (2024) · Discover Education · doi:10.1007/s44217-024-00138-2 · Cited by 191 · see citing works
A scoping review of ChatGPT research in accounting and finance
Dong, M., Stratopoulos, T. C., & Wang, V. X. (2024) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2024.100715 · Cited by 160 · see citing works
The impact of generative AI on information processing: Evidence from the ban of ChatGPT in Italy
Bertomeu, J., Lin, Y., Liu, Y., & Ni, Z. (2025) · Journal of Accounting and Economics · doi:10.1016/j.jacceco.2025.101782 · Cited by 74 · see citing works
Rane, N. L. (2024) · International Journal of Artificial Intelligence and Machine Learning · doi:10.51483/ijaiml.4.1.2024.22-47 · Cited by 53 · see citing works
Vision-Enabled Large Language and Deep Learning Models for Image-Based Emotion Recognition
Nadeem, M., Sohail, S. S., Javed, L., Anwer, F., Saudagar, A. K. J., & Muhammad, K. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10281-5 · Cited by 39 · see citing works
Gezdur, A., & Bhattacharjya, J. (2025) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/ijpdlm-12-2023-0492 · Cited by 37 · see citing works
Güler, N. K., Dertli, Z. G., Boran, E., & Yıldız, B. (2024) · Pedagogical Research · doi:10.29333/pr/14145 · Cited by 54 · see citing works
Exploring the competence of ChatGPT for customer and patient service management
Haleem, A., Javaid, M., & Singh, R. P. (2024) · Intelligent Pharmacy · doi:10.1016/j.ipha.2024.03.002 · Cited by 38 · see citing works
A Systematic Literature Review of Empirical Research on ChatGPT in EducationPreprint
Albadarin, Y., Tukiainen, M., Saqr, M., & Pope, N. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4562771 · Cited by 26 · see citing works
Bus Bunching and Bus Bridging: What Can We Learn from Generative AI Tools like ChatGPT?
Voß, S. (2023) · Sustainability · doi:10.3390/su15129625 · Cited by 25 · see citing works
Pre-made Empowering Artificial Intelligence and ChatGPT: The Growing Importance of Human AI-Experts
Virvou, M. K., & Tsihrintzis, G. A. (2023) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa59645.2023.10345880 · Cited by 22 · see citing works
An analysis of Watson vs. BARD vs. ChatGPT: The Jeopardy! Challenge
O’Leary, D. E. (2023) · AI Magazine · doi:10.1002/aaai.12118 · Cited by 17 · see citing works
Self-citationEnterprise Large Language Models: Knowledge Characteristics, Risks and Organizational Activities
O’Leary, D. E. (2023) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1541 · Cited by 17 · see citing works
Self-citationSuccessfully Using ChatGPT in Logistics: Are We There Yet?
Voß, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43612-3_1 · Cited by 15 · see citing works
Generative Artificial Intelligence in the Classroom: A Financial Accounting Experience
Calderon, T. G., Gao, L., & Cardoso, R. L. (2023) · Emerald Group Publishing Limited eBooks · doi:10.1108/s1085-462220230000027006 · Cited by 8 · see citing works
Mabwe, K., Aminu, N., Ivanov, S. H., & Dimov, D. (2025) · foresight · doi:10.1108/fs-06-2024-0122 · Cited by 8 · see citing works
Using large language models to write theses and dissertations
O’Leary, D. E. (2023) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1547 · Cited by 6 · see citing works
Self-citationNadeem, M., Sohail, S. S., Madsen, D. Ø., Alzahrani, A. I., Ser, J. D., & Muhammad, K. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536937 · Cited by 6 · see citing works
Using Machine Learning to Generate a Dictionary for Environmental Issues
O’Leary, D. E., & Yoon, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40837-3_9 · Cited by 6 · see citing works
Self-citationPerformance of chatbots in queries concerning fundamental concepts in photochemistry
Taniguchi, M., & Lindsey, J. S. (2024) · Photochemistry and Photobiology · doi:10.1111/php.14037 · Cited by 6 · see citing works
AI for Good: History, Open Data and Some ESG-based Applications
O’Leary, D. E. (2025) · Journal of Decision System · doi:10.1080/12460125.2024.2443182 · Cited by 5 · see citing works
Self-citationQualitative Data Analysis in the Age of Artificial General Intelligence
Abdüsselam, M. S. (2023) · International Journal of Advanced Natural Sciences and Engineering Researches · doi:10.59287/ijanser.454 · Cited by 4 · see citing works
Generative AI in Academic Settings: Exploring ChatGPT Adoption and Implications
Lima, G. M. L., Sanchez, L. B. V., Pereira, J. D. S., Alcântara, V. D. C., & Christino, J. M. M. (2024) · Journal of Education and Learning · doi:10.5539/jel.v13n4p144 · Cited by 3 · see citing works
Is ChatGPT an Accurate Source of Information for Uninformed Taxpayers?
Antinozzi, H. S., & Cooper, L. A. (2025) · Journal of Emerging Technologies in Accounting · doi:10.2308/jeta-2023-072 · Cited by 6 · see citing works
The Role of Chatbots in Enquiry‐Based Learning for Oral Health Students—An Exploratory Study
Khareedi, R., & Fernandez, D. (2025) · European Journal Of Dental Education · doi:10.1111/eje.13115 · Cited by 3 · see citing works
A Review on Conversational AI as a Tool in Academic Writing
Demiröz, H., & Tıkız‐Ertürk, G. (2025) · Eskiyeni (Online)/Eskiyeni · doi:10.37697/eskiyeni.1565854 · Cited by 4 · see citing works
Ecosystem for Construction of Hybrid Conversational Systems (BRasa)
Mir, Y. O. V., Pupo, I. P., Pérez, P. Y. P., Acuña, L. A., & Bello, R. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-50495-2_8 · Cited by 2 · see citing works
Professional Work in the Era of Chat GPT
Tarasiuk, G., & Czapski, G. (2023) · EUROPEAN RESEARCH STUDIES JOURNAL · doi:10.35808/ersj/3223 · Cited by 1 · see citing works
A Benchmark Comparison of a Domain-Focused Pipeline with ChatGPT
Zhang, C., Zhao, T., Saraoğlu, H., & Louton, D. A. (2025) · Journal of Computer Information Systems · doi:10.1080/08874417.2024.2446629 · Cited by 1 · see citing works
Using Large Language Models for Armchair Auditors
O’Leary, D. E. (2024) · Digital Government Research and Practice · doi:10.1145/3676280
Self-citationQiu, Y., & Meng, Y. (2025) · Digital Library Perspectives · doi:10.1108/dlp-02-2025-0029
The Evolving World of Enterprise Knowledge Management: History, Implications, and Emerging Issues
O’Leary, D. E., French, A. M., & Storey, V. C. (2026) · Knowledge management and organizational learning · doi:10.1007/978-3-032-14721-9_4
Self-citationAlles, M. G., & Gal, G. (2026) · Journal of Emerging Technologies in Accounting · doi:10.2308/jeta-2024-022
Miyamoto, T. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100375
Emruli, B., Geerts, G., & O’Leary, D. E. (2026) · Journal of Information Technology Teaching Cases · doi:10.1177/20438869261489086
Self-citation
- Authors
- O'Leary, D. E.
- Affiliations
- University of Southern California Los Angeles California USA
- Venue
- Intelligent Systems in Accounting, Finance and Management
- Publisher
- Wiley
- Published
- 2023-01
- Volume
- 30
- Issue
- 1
- Pages
- 41-54
- ISSN
- 1055-615X (ISSN-L)
- 1550-1949
- 2160-0074
- OpenAlex
- W4361858653
- Cited by
- 38
- References
- 10
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 10 · 2023: 13
- FWCI
- 3.36
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management (publishedVersion, public-domain)
- SSRN Electronic Journal (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Keywords
- chatbots, ChatGPT, LaMDA, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
