Understanding User Perception of Automated News Generation System
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Automated journalism refers to the generation of news articles using computer programs. Although it is widely used in practice, its user experience and interface design remain largely unexplored. To understand the user perception of an automated news system, we designed NewsRobot, a research prototype that automatically generated news on major events of the PyeongChang 2018 Winter Olympic Games in real-time. It produces six types of news by combining two kinds of content (general/individualized) and three styles (text, text+image, text+image+sound). A total of 30 users participated in using NewsRobot, completing surveys and interviews on their experience. Our findings are as follows: (1) Users preferred individualized news yet considered it less credible, (2) more presentation elements were appreciated but only if their quality was assured, and (3) NewsRobot was considered factual and accurate yet shallow in depth. Based on our findings, we discuss implications for designing automated journalism user interfaces.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Filter Bubbles, Echo Chambers, and Online News Consumption
Flaxman, S., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw006 · Cited by 1,992
Accountability in algorithmic decision making
Diakopoulos, N. (2016) · Communications of the ACM · doi:10.1145/2844110 · Cited by 763
Algorithmic Accountability: Journalistic investigation of computational power structures
Diakopoulos, N. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976411 · Cited by 733
Bias in algorithmic filtering and personalization
Bozdag, E. (2013) · Ethics and Information Technology · doi:10.1007/s10676-013-9321-6 · Cited by 649
Carlson, M. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976412 · Cited by 542
Advancing Tailored Health Communication: A Persuasion and Message Effects Perspective
Rimer, B. K., & Kreuter, M. W. (2006) · Journal of Communication · doi:10.1111/j.1460-2466.2006.00289.x · Cited by 531 · cited 1×
“Research has demonstrated that personalized messages can engage and persuade an audience more effectively than generic mass messages [44, 45].”
BackgroundVeale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1×
“Although not explicitly addressing automated journalism, recent discussions on the transparency and fairness of algorithms in automated systems show that it is important to not only focus on the technical design of the algorithm but also closely observe the user experience of these systems [56, 61].”
BackgroundEnter the Robot Journalist: Users' perceptions of automated content
Clerwall, C. (2014) · Journalism Practice · doi:10.1080/17512786.2014.883116 · Cited by 377 · cited 1×
“Considering that software-generated content is usually perceived as descriptive and boring, despite being considered objective [10], this could be an inherent problem that automotive journalism needs to address.”
BackgroundA comparison of static, adaptive, and adaptable menus
Findlater, L., & McGrenere, J. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985704 · Cited by 316 · cited 1×
“In this situation, introducing the concept of adaptable interfaces in designing automation systems [24] could be considered.”
BackgroundReaders’ perception of computer-generated news: Credibility, expertise, and readability
Graefe, A., Haim, M., Haarmann, B., & Brosius, H.-B. (2018) · Journalism · doi:10.1177/1464884916641269 · Cited by 315
A Qualitative Exploration of Perceptions of Algorithmic Fairness
Woodruff, A., Fox, S. E., Rousso-Schindler, S., & Warshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174230 · Cited by 256 · cited 1×
“Although not explicitly addressing automated journalism, recent discussions on the transparency and fairness of algorithms in automated systems show that it is important to not only focus on the technical design of the algorithm but also closely observe the user experience of these systems [56, 61].”
BackgroundLokot, T. (2016) · doi:10.1080/21670811.2015.1081822 · Cited by 209 · cited 1×
“Since this technology can create a large number of news articles quickly and accurately, it has attracted the attention of both the media industry [22, 31, 48] and scholars [8, 36, 52].”
BackgroundMontal, T., & Reich, Z. (2017) · Digital Journalism · doi:10.1080/21670811.2016.1209083 · Cited by 208
Graefe, A. (2016) · Columbia Academic Commons (Columbia University) · doi:10.7916/d80g3xdj · Cited by 186
Automating the News: How Personalized News Recommender System Design Choices Impact News Reception
Beam, M. A. (2014) · Communication Research · doi:10.1177/0093650213497979 · Cited by 159 · cited 1×
“On the other hand, the use of a personalized news system can have a direct negative effect on knowledge acquisition [3].”
BackgroundSupporting the Use of User Generated Content in Journalistic Practice
Tolmie, P., Procter, R., Randall, D. W., Rouncefield, M., Burger, C., Wong Sak Hoi, G., Zubiaga, A., & Liakata, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025892 · Cited by 72 · cited 1×
“Several studies have proposed different kinds of authoring tools to support news content generation [1, 37, 53].”
Background
Cited by
Large-scale Text-to-Image Generation Models for Visual Artists’ Creative Works
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584078 · Cited by 189
Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information Seeking
Sharma, N., Liao, Q. V., & Xiao, Z. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642459 · Cited by 132
AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580907 · Cited by 98
“Within the frst category, past work has examined how news is shared [52], specifcally the role search engines play in biasing the content we consume [55], as well as how users interact with and consume news [6] [22] [41].”
BackgroundKim, T., Shin, D., Kim, Y.-H., & Hong, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642693 · Cited by 75
“As people increasingly adopt LLMs for writing, understanding user perception and experience of writing with LLMs has become an emerging area of interest in the Human-Computer Interaction (HCI) community [8, 33, 46].”
Metadata
- Authors
- Oh, C., Choi, J., Lee, S., Park, S., Kim, D., Song, J., Kim, D., Lee, J., & Suh, B.
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-13
- OpenAlex
- W3030174958
- Cited by
- 32
- References
- 18
- FWCI
- 14.71
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.