The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We measure the originality of each AUT idea with an existing fine-tuned GPT-3 classifier [44].”
From Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking (2025) · cites this paper as [44]
In the citing paper
“We chose these five particular objects as the originality scoring measure had the highest correlation with human judgements for these five objects [44].”
From Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking (2025) · cites this paper as [44]
In the citing paper
“The model was fine-tuned with human judgments of AUT originality (where human raters judged, on a scale of 1 to 5, the originality of an idea given the object) and achieved an 𝑟 = 0.81 overall correlation with human judgments.”
From Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking (2025) · cites this paper as [44]
In the citing paper
“we employ the Alternate Uses Task (AUT; Organisciak et al. (2023)), an established measurement for divergent (as in free-associative) thinking, which possesses a validated and fully automated evaluation pipeline (Organisciak et al., 2025).”
From Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks (2026) · cites this paper as Organisciak et al., 2023
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 14,137
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,677
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Guilford, J. P. (1950) · American Psychologist · doi:10.1037/h0063487 · Cited by 3,588
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103
XAI—Explainable artificial intelligence
Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.-Z. (2019) · Science Robotics · doi:10.1126/scirobotics.aay7120 · Cited by 2,053
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Divergent Thinking as an Indicator of Creative Potential
Runco, M. A., & Acar, S. (2012) · Creativity Research Journal · doi:10.1080/10400419.2012.652929 · Cited by 1,696
Self-citationSuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051
Mapping authors in intellectual space: A technical overview
McCain, K. W. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6 · Cited by 987
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929
Silvia, P. J., Winterstein, B. P., Willse, J. T., Barona, C. M., Cram, J. T., Hess, K. I., Martinez, J. L., & Richard, C. A. (2008) · Psychology of Aesthetics, Creativity, and the Arts · doi:10.1037/1931-3896.2.2.68 · Cited by 912
Robust prediction of individual creative ability from brain functional connectivity
Beaty, R. E., Kenett, Y. N., Christensen, A. P., Rosenberg, M. D., Benedek, M., Chen, Q., Fink, A., Qiu, J., Kwapil, T. R., Kane, M. J., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1713532115 · Cited by 835
Can We Trust Creativity Tests? A Review of the Torrance Tests of Creative Thinking (TTCT)
Kim, K. H. (2006) · Creativity Research Journal · doi:10.1207/s15326934crj1801_2 · Cited by 830
Veale, M., & Zuiderveen Borgesius, F. (2021) · Computer Law Review International · doi:10.9785/cri-2021-220402 · Cited by 805
Text and Code Embeddings by Contrastive Pre-TrainingPreprint
Neelakantan, A., Xu, T., Puri, R., Radford, A., Han, J. M., Tworek, J., Yuan, Q.-M., Tezak, N., Kim, J. W., Hallacy, C., et al. (2022) · arXiv.org · doi:10.48550/arxiv.2201.10005 · Cited by 661
Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models
Ni, J., Ábrego, G. H., Constant, N., Ma, J., Hall, K., Cer, D. M., & Yang, Y. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.146 · Cited by 630
Beaty, R. E., & Silvia, P. J. (2012) · Psychology of Aesthetics, Creativity, and the Arts · doi:10.1037/a0029171 · Cited by 551
Modes of Thinking in Young Children
Mackie, J. (1966) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-196604000-00018 · Cited by 516
Predictive Validity of the Torrance Tests of Creative Thinking*
TORRANCE, E. P. (1972) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.1972.tb00936.x · Cited by 516
Silvia, P. J., Nusbaum, E. C., Berg, C., Martin, C. L., & O’Connor, A. (2009) · Journal of Research in Personality · doi:10.1016/j.jrp.2009.04.015 · Cited by 451
Runco, M. A., Millar, G. W., Acar, S., & Cramond, B. L. (2010) · Creativity Research Journal · doi:10.1080/10400419.2010.523393 · Cited by 397
Self-citationKim, K. H. (2008) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.2008.tb01290.x · Cited by 378
Automating creativity assessment with SemDis: An open platform for computing semantic distance
Beaty, R. E., & Johnson, D. R. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01453-w · Cited by 341
Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning
Roemmele, M., Bejan, C. A., & Gordon, A. S. (2011) · Cited by 334
Cramond, B., Matthews-Morgan, J., Bandalos, D., & Zuo, L. (2005) · Gifted Child Quarterly · doi:10.1177/001698620504900402 · Cited by 318
Downie, J. S. (2008) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.29.247 · Cited by 297
Benedek, M., Mühlmann, C., Jauk, E., & Neubauer, A. C. (2013) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0033644 · Cited by 238
Scoring Divergent Thinking Tests Using Total Ideational Output and a Creativity Index
Runco, M. A., & Mraz, W. (1992) · Educational and Psychological Measurement · doi:10.1177/001316449205200126 · Cited by 194
Understanding Fluency and Originality: A latent variable perspective
Dumas, D., & Dunbar, K. N. (2014) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2014.09.003 · Cited by 188
Self-citationDumas, D., Organisciak, P., & Doherty, M. (2021) · Psychology of Aesthetics, Creativity, and the Arts · doi:10.1037/aca0000319 · Cited by 158
Self-citationSilvia, P. J. (2010) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2010.06.001 · Cited by 142
Leakage and the Reproducibility Crisis in ML-based SciencePreprint
Kapoor, S., & Narayanan, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.07048 · Cited by 140
Which Test of Divergent Thinking Is Best?
Runco, M., Abdulla, A., Paek, S., Al-Jasim, F., & Alsuwaidi, H. N. (2016) · Creativity Theories – Research – Applications · doi:10.1515/ctra-2016-0001 · Cited by 135
Johnson, D. R., Kaufman, J. C., Baker, B. S., Patterson, J. D., Barbot, B., Green, A. E., van Hell, J. G., Kennedy, E., Sullivan, G., Taylor, C. L., et al. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01986-2 · Cited by 118
Creativity measurement in undergraduate students from 1984–2013: A systematic review.
Snyder, H. T., Hammond, J. A., Grohman, M. G., & Katz-Buonincontro, J. L. (2019) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/aca0000228 · Cited by 98
Application of Latent Semantic Analysis to Divergent Thinking is Biased by Elaboration
Forthmann, B., Oyebade, O., Ojo, A., Günther, F., & Holling, H. (2018) · The Journal of Creative Behavior · doi:10.1002/jocb.240 · Cited by 92
Divergent thinking and creative achievement—How strong is the link? An updated meta-analysis.
Said‐Metwaly, S., Taylor, C. L., Camarda, A., & Barbot, B. (2022) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/aca0000507 · Cited by 89
Runco, M. A. (2008) · Cited by 83
Applying Automated Originality Scoring to the Verbal Form of Torrance Tests of Creative Thinking
Acar, S., Berthiaume, K., Grajzel, K., Dumas, D., Flemister, C. “., & Organisciak, P. (2021) · Gifted Child Quarterly · doi:10.1177/00169862211061874 · Cited by 69
Self-citationOld or New? Evaluating the Old/New Scoring Method for Divergent Thinking Tasks
Silvia, P. J., Nusbaum, E. C., & Beaty, R. E. (2015) · The Journal of Creative Behavior · doi:10.1002/jocb.101 · Cited by 68
Buczak, P., Huang, H., Forthmann, B., & Doebler, P. (2022) · The Journal of Creative Behavior · doi:10.1002/jocb.559 · Cited by 46
On the Dependability and Feasibility of Layperson Ratings of Divergent Thinking
Hass, R. W., Rivera, M., & Silvia, P. J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01343 · Cited by 41
Zaccaro, S. J., Connelly, S., Repchick, K. M., Daza, A. I., Young, M. C., Kilcullen, R. N., Gilrane, V. L., Robbins, J. M., & Bartholomew, L. (2015) · The Leadership Quarterly · doi:10.1016/j.leaqua.2015.03.007 · Cited by 38
Acar, S. (2022) · Creativity Research Journal · doi:10.1080/10400419.2022.2044656 · Cited by 31
Self-citationForthmann, B., & Doebler, P. (2022) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/aca0000518 · Cited by 29
Mohr, A. H., Sell, A., & Lindsay, T. A. (2015) · Social Science Computer Review · doi:10.1177/0894439315588736 · Cited by 23
Handbook of Creativity Assessment
(2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781839102158 · Cited by 22
Shaw, A. (2021) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2021.100789 · Cited by 18
What Makes Children's Responses to Creativity Assessments Difficult to Judge Reliably?
Dumas, D., Acar, S., Berthiaume, K., Organisciak, P., Eby, D., Grajzel, K., Vlaamster, T., Newman, M., & Carrera, M. (2023) · The Journal of Creative Behavior · doi:10.1002/jocb.588 · Cited by 16
Self-citationHow do the kids speak? Improving educational use of text mining with child-directed language models
Organisciak, P., Newman, M., Eby, D., Acar, S., & Dumas, D. (2023) · Information and Learning Sciences · doi:10.1108/ils-06-2022-0082 · Cited by 14
Self-citationScoring Creativity Tests By Computer
Paulus, D. H., & Renzuli, J. S. (1968) · Gifted Child Quarterly · doi:10.1177/001698626801200202 · Cited by 11
Ji, Y., & Gao, S. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 10
Stevenson, C. S., Smal, I., Baas, M., Dahrendorf, M., Grasman, R. P. P. P., Tanis, C. C., Scheurs, E., Sleiffer, D., & van der Maas, H. L. J. (2020) · Data Archiving and Networked Services (DANS) · Cited by 8
Forthmann, B., & Doebler, P. (2022) · doi:10.31234/osf.io/byj8c · Cited by 5
Divergent thinking: early views
Plucker, J. A., Meyer, M. S., & Liu, P. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781839102158.00015 · Cited by 5
Research works printed but not linked: 1
- Torrance, 1980, Growing up creatively gifted: A 22-yr longitudinal study, Creative Child & Adult Quarterly, 5, 148
Cited by
Sorted by citations
Artificial Intelligence & Creativity: A Manifesto for Collaboration
Vinchon, F., Lubart, T., Bartolotta, S., Gironnay, V., Botella, M., Bourgeois-Bougrine, S., Burkhardt, J., Bonnardel, N., Corazza, G. E., Glăveanu, V., et al. (2023) · The Journal of Creative Behavior · doi:10.1002/jocb.597 · Cited by 273
Artificial muses: Generative artificial intelligence chatbots have risen to human-level creativity
Haase, J., & Hanel, P. H. P. (2023) · Journal of Creativity · doi:10.1016/j.yjoc.2023.100066 · Cited by 233
Applying large language models and chain-of-thought for automatic scoring
Lee, G.-G., Latif, E., Wu, X., Liu, N., & Zhai, X. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100213 · Cited by 186
Fine-tuning ChatGPT for automatic scoring
Latif, E., & Zhaı, X. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100210 · Cited by 244
The originality of machines: AI takes the Torrance test
Guzik, E. E., Byrge, C., & Gilde, C. (2023) · Journal of Creativity · doi:10.1016/j.yjoc.2023.100065 · Cited by 165
Metadata
- Authors
- Organisciak, P., Acar, S., Dumas, D., & Berthiaume, K.
- Affiliations
- University of Denver (US)
- University of Georgia (US)
- University of North Texas (US)
- Corresponding author
- Organisciak, Peter
- Venue
- Thinking Skills and Creativity
- Publisher
- Elsevier BV
- Published
- 2023-09
- Volume
- 49
- Pages
- 101356
- ISSN
- 1871-1871 (ISSN-L)
- OpenAlex
- W4381487157
- Cited by
- 202
- References
- 80
- Citations by year
- 2026: 65 · 2025: 64 · 2024: 56 · 2023: 16
- FWCI
- 52.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Thinking Skills and Creativity (publishedVersion)
- Funders
- Institute of Education Sciences (305A200199)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Keywords
- divergent thinking, automated grading, alternative uses task, large language models, creativity assessment, semantic distance, fine-tuning, transfer learning, few-shot prompting, zero-shot prompting, human judgment correlation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
