AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Open accessAbstract
News media often leverage documents to find ideas for stories, while being critical of the frames and narratives present. Developing angles from a document such as a press release is a cognitively taxing process, in which journalists critically examine the implicit meaning of its claims. Informed by interviews with journalists, we developed AngleKindling, an interactive tool which employs the common sense reasoning of large language models to help journalists explore angles for reporting on a press release. In a study with 12 professional journalists, we show that participants found AngleKindling significantly more helpful and less mentally demanding to use for brainstorming ideas, compared to a prior journalistic angle ideation tool. AngleKindling helped journalists deeply engage with the press release and recognize angles that were useful for multiple types of stories. From our findings, we discuss how to help journalists customize and identify promising angles, and extending AngleKindling to other knowledge-work domains.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · see citing works
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Design Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639 · cited 1× · see citing works
“Generative models are being successfully applied to support a number of creative tasks, including music composition [38], designing visual art [36] [35], and writing [24] [14] [47].”
MethodSelf-citationMarcus, A., Bernstein, M. S., Badar, O., Karger, D. R., Madden, S. R., & Miller, R. C. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978975 · Cited by 543 · see citing works
Wu, T., Terry, M. A., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 536 · see citing works
Wordcraft: Story Writing With Large Language Models
Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511105 · Cited by 374 · see citing works
Toward Automated Fact-Checking
Hassan, N., Arslan, F., Li, C., & Tremayne, M. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098131 · Cited by 303 · see citing works
The Handbook of Journalism Studies
Meijer, I. C., Wahl‐Jorgensen, K., & Hanitzsch, T. (2019) · Routledge eBooks · doi:10.4324/9781315167497 · Cited by 286 · see citing works
Jigsaw: Supporting Investigative Analysis through Interactive Visualization
Stasko, J., Görg, C., & Liu, Z. (2008) · Information Visualization · doi:10.1057/palgrave.ivs.9500180 · Cited by 281 · see citing works
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271 · see citing works
Diamonds in the rough: Social media visual analytics for journalistic inquiry
Diakopoulos, N. A., Naaman, M., & Kivran-Swaine, F. (2010) · IEEE Symposium on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2010.5652922 · Cited by 215 · see citing works
Self-citationCohen, S., Hamilton, J. T., & Turner, F. (2011) · Communications of the ACM · doi:10.1145/2001269.2001288 · Cited by 208 · see citing works
PromptChainer: Chaining Large Language Model Prompts through Visual Programming
Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519729 · Cited by 198 · see citing works
Falling for Fake News: Investigating the Consumption of News via Social Media
Flintham, M., Karner, C., Bachour, K., Creswick, H., Gupta, N., & Moran, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173950 · Cited by 194 · see citing works
The American Journalist in the Digital Age: Another Look at U.S. News People
Weaver, D. H., Willnat, L., & Wilhoit, G. C. (2018) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699018778242 · Cited by 175 · see citing works
Yu, L., & Nickerson, J. V. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979147 · Cited by 169 · see citing works
Self-citationSparks: Inspiration for Science Writing using Language Models
Gero, K. I., Liu, V., & Chilton, L. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533533 · Cited by 161 · see citing works
Self-citationPromptMaker: Prompt-based Prototyping with Large Language Models
Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503564 · Cited by 157 · cited 2× · see citing works
“Finally, LLMs are also currently being used to enable end-users to develop their own AI-infused applications in the form of LLM-chains, where the output of one LLM-step is fed into another [60] [59] [27].”
BackgroundMetaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146 · see citing works
Self-citationFinding and assessing social media information sources in the context of journalism
Diakopoulos, N., De Choudhury, M., & Naaman, M. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208409 · Cited by 144 · see citing works
Self-citationWang, H., Cosley, D., & Fussell, S. R. (2010) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1718918.1718938 · Cited by 138 · see citing works
Toward Collaborative Ideation at Scale
Siangliulue, P., Arnold, K. C., Gajos, K. Z., & Dow, S. P. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675239 · Cited by 127 · see citing works
Where is Your Evidence: Improving Fact-checking by Justification Modeling
Alhindi, T., Petridis, S., & Muresan, S. (2018) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/w18-5513 · Cited by 125 · see citing works
Self-citationAn Empirical Survey on Long Document Summarization: Datasets, Models, and Metrics
Koh, H. Y., Ju, J., Liu, M., & Pan, S. (2022) · ACM Computing Surveys · doi:10.1145/3545176 · Cited by 124 · see citing works
Trielli, D., & Diakopoulos, N. A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300683 · Cited by 111 · see citing works
Self-citationOpal: Multimodal Image Generation for News Illustration
Liu, V., Qiao, H., & Chilton, L. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545621 · Cited by 104 · see citing works
Self-citationSiangliulue, P., Chan, J., Dow, S. P., & Gajos, K. Z. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984511.2984578 · Cited by 99 · see citing works
THE PROCESS MODEL OF NEWS INITIATIVE
Reich, Z. (2006) · Journalism Studies · doi:10.1080/14616700600757928 · Cited by 97 · see citing works
Diakopoulos, N. A. (2020) · Digital Journalism · doi:10.1080/21670811.2020.1736946 · Cited by 91 · see citing works
Self-citationBrehmer, M., Ingram, S., Stray, J., & Munzner, T. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346431 · Cited by 90 · see citing works
Xia, C., Schwartz, R., Xie, K. E., et al. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2567948.2577020 · Cited by 81 · see citing works
MetaMap: Supporting Visual Metaphor Ideation through Multi-dimensional Example-based Exploration
Kang, Y., Sun, Z., Wang, S., Huang, Z., Wu, Z., & Ma, X. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445325 · Cited by 81 · see citing works
Jiang, E., Toh, E., Molina, A., Olson, K., Kayacik, C., Donsbach, A., Cai, C. J., & Terry, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501870 · Cited by 79 · see citing works
Osone, H., Lu, J.-L., & Ochiai, Y. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3450391 · Cited by 74 · cited 1× · see citing works
“BunCho, also uses an LLM to generate titles and synopses from keywords [43].”
MethodSAGA: Collaborative Storytelling with GPT-3
Shakeri, H., Neustaedter, C., & DiPaola, S. (2021) · Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3462204.3481771 · Cited by 66 · see citing works
Sourcing the News: Key Issues in Journalism - an Innovative Study of the Israeli Press
Örnebring, H. (2010) · Journalism & Mass Communication Quarterly · Cited by 60 · see citing works
Shi, Y., Wang, Y., Qi, Y., Chen, J., Xu, X., & Ma, K. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998208 · Cited by 57 · see citing works
Chang, M., Guillain, L. V., Jung, H., et al. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174025 · Cited by 55 · see citing works
Understanding Online News Behaviors
Bentley, F. R., Quehl, K., Wirfs-Brock, J., & Bica, M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300820 · Cited by 55 · see citing works
Reuters tracer: Toward automated news production using large scale social media data
Liu, X., Nourbakhsh, A., Li, Q., Shah, S., Martin, R., & Duprey, J. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258082 · Cited by 55 · see citing works
Maiden, N., Zachos, K., Brown, A., Brock, G., Nyre, L., Tonheim, A. N., Apsotolou, D., & Evans, J. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174049 · Cited by 53 · see citing works
Andolina, S., Klouche, K., Cabral, D., Ruotsalo, T., & Jacucci, G. (2015) · Creativity and Cognition · doi:10.1145/2757226.2757252 · Cited by 48 · see citing works
Semantically Far Inspirations Considered Harmful?
Chan, J., Siangliulue, P., McDonald, D. Q., Liu, R., Moradinezhad, R., Aman, S., Solovey, E., Gajos, K. Z., & Dow, S. P. (2017) · Creativity and Cognition · doi:10.1145/3059454.3059455 · Cited by 48 · see citing works
Andolina, S., Schneider, H., Chan, J., Klouche, K., Jacucci, G., & Dow, S. P. (2017) · Creativity and Cognition · doi:10.1145/3059454.3059477 · Cited by 46 · see citing works
Language models: past, present, and future
Li, H. (2022) · Communications of the ACM · doi:10.1145/3490443 · Cited by 45 · see citing works
Boumans, J. W. (2017) · Journalism Studies · doi:10.1080/1461670x.2017.1338154 · Cited by 36 · see citing works
Diakopoulos, N. A., Trielli, D., & Lee, G. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479550 · Cited by 35 · see citing works
Self-citationUnderstanding User Perception of Automated News Generation System
Oh, C., Choi, J., Lee, S., Park, S., Kim, D., Song, J., Kim, D., Lee, J., & Suh, B. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376811 · Cited by 34 · see citing works
Diakopoulos, N. A., Goldenberg, S., & Essa, I. A. (2009) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1518701.1518824 · Cited by 30 · see citing works
Self-citationSpinneret: Aiding Creative Ideation through Non-Obvious Concept Associations
Bae, S. L., Kwon, O.-H., Chandrasegaran, S. K., & Ma, K. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376746 · Cited by 30 · see citing works
Wang, Y., & Diakopoulos, N. A. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445266 · Cited by 22 · see citing works
Self-citationIdeateRelate: An Examples Gallery That Helps Creators Explore Ideas in Relation to Their Own
Xu, X. (., Xiong, R., Wang, B., Min, D., & Dow, S. P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479496 · Cited by 21 · see citing works
Disseminating Research News in HCI
Smith, C. E., Nevarez, E., & Zhu, H. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376744 · Cited by 20 · see citing works
Chan, J., Dang, S., & Dow, S. P. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818052.2874313 · Cited by 18 · see citing works
TaleBrush: Visual Sketching of Story Generation with Pretrained Language Models
Chung, J. J. Y., Kim, W., Yoo, K. M., Lee, H., Adar, E., & Chang, M. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519873 · Cited by 18 · see citing works
Huang, G., & Quinn, A. J. (2017) · Creativity and Cognition · doi:10.1145/3059454.3059481 · Cited by 16 · see citing works
August, T., Wang, L. L., Bragg, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.00130 · Cited by 16 · see citing works
Kita, Y., & Rekimoto, J. (2018) · Augmented Human International Conference · doi:10.1145/3174910.3174937 · Cited by 8 · see citing works
SymbolFinder: Brainstorming Diverse Symbols Using Local Semantic Networks
Petridis, S., Shin, H. V., & Chilton, L. B. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474757 · Cited by 8 · see citing works
Self-citationPublic Relations and Journalism
Sherwood, M., Marjoribanks, T., & Nicholson, M. (2019) · Oxford Research Encyclopedia of Communication · doi:10.1093/acrefore/9780190228613.013.866 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Pra- fulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jefrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877-1901. https://proceedings. neurips.cc/paper/2020/fle/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
- Robert M Entman. 1993. Framing: Towards clarifcation of a fractured paradigm. McQuail's reader in mass communication theory 390 (1993), 397.
- Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards Understanding and Mitigating Social Biases in Language Models. In Proceedings of the 38th International Conference on Machine Learning (Proceed- ings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 6565-6576. https://proceedings.mlr.press/v139/liang21a.html
Cited by
Sorted by citations
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570 · see citing works
“Much of the recent work on generative AI with human participants, however, tends to focus on attitudes towards the tool 44,45 , interviews or think-alouds with participants 46–48 , or user experience instead of task performance 49–51 .”
Large language models show human-like content biases in transmission chain experiments
Acerbi, A., & Stubbersfield, J. M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2313790120 · Cited by 137 · see citing works
Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
Dhillon, P. S., Molaei, S., Li, J., Golub, M., Zheng, S., & Robert, L. P. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642134 · Cited by 111 · see citing works
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Agarwal, D., Naaman, M., & Vashistha, A. (2024) · arXiv · doi:10.48550/arxiv.2409.11360 · Cited by 99
When and How to Use AI in the Design Process? Implications for Human-AI Design Collaboration
Lee, S.-Y., Law, M., & Hoffman, G. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2353451 · Cited by 97 · see citing works
Kim, T. S., Lee, Y., Chang, M., & Kim, J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606833 · Cited by 56 · see citing works
PANDALens: Towards AI-Assisted In-Context Writing on OHMD During Travels
Cai, R., Janaka, N., Chen, Y., Wang, L., Zhao, S., & Liu, C. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642320 · Cited by 36 · see citing works
Wang, J., Hu, H., Wang, Z., Yan, S., Sheng, Y., & He, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641917 · Cited by 21 · see citing works
- Authors
- Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B.
- Affiliations
- Columbia University (US)
- Northwestern University (US)
- Syracuse University (US)
- Stevens Institute of Technology (US)
- ORCID
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-16
- OpenAlex
- W4366729084
- Cited by
- 98
- References
- 62
- Citations by year
- 2026: 13 · 2025: 32 · 2024: 47 · 2023: 6
- FWCI
- 141.67
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- NSF (National Science Foundation) (1909803,2113906,2129047,2128906,2129020)
- Topics
- Social Sciences › Communication › Journalism and Media Studies
- Social Sciences › Communication › Media Studies and Communication
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- large language models, press releases, news framing, commonsense reasoning, knowledge work
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
