CoChat

Retraining fact-checkers: The emergence of ChatGPT in information verification

Cuartielles, Roger, Ramon-Vegas, Xavier, Pont-Sorribes, Carles · 2023 · El Profesional de la información
Cited by 25

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Abstract

The open launch of new artificial intelligence tools such as ChatGPT-3.5 (Generated Pre-trained Transformer) in November 2022 by the company OpenAI -and then its update to version GPT-4 in March 2023- poses new opportunities and challenges for journalism, and especially for professionals specifically focused on information verification. This research aims to understand and analyze the perceptions generated by the irruption of ChatGPT among fact-checking professionals in Spain with the aim of identifying disadvantages and advantages in its use, professional implications and desired functionalities. The study uses qualitative methodology with in-depth interviews with professionals from all Spanish fact-checking platforms belonging to the International Factchecking Network (IFCN) and the European Digital Media Observatory (EDMO). The results conclude that the use of ChatGPT presents notable ambivalences. On the one hand, there are perceived drawbacks in issues such as the transparency and reliability of sources, the scope of the data, and the format of the responses generated. However, fact-checkers also point to a possible auxiliary use of the chatbot in the tasks of gathering information, detecting falsehoods, and producing denials. The irruption of ChatGPT has a direct impact on the work routines of the fact-checkers, which can be made more difficult, reinforced or extended. Fact-checking professionals perceive themselves as “context agents” in a new ecosystem that also obliges them to further diversify their fields of action in the fight against disinformation and to accelerate the implementation of media education actions that empower citizens in the responsible use of artificial intelligence.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3145/epi.2023.sep.15

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 61

  1. Arias-Jiménez, Bryan; Rodríguez-Hidalgo, Claudia; Mier-Sanmartín, Catalina; Coronel-Salas, Gabriela (2023). “Use of chatbots for news verification”. In: López-López, Paulo-Carlos; Barredo, Daniel; Torres-Toukoumidis, Ángel; De-Santis, Andrea; Avilés, Óscar (eds.). Communication and applied technologies. Smart innovation, systems and technologies. Singapur: Springer Nature, v. 318, pp. 133-143. ISBN: 978 981 19 6347 6 https://doi.org/10.1007/978-981-19-6347-6_12 · doi:10.1007/978-981-19-6347-6
  2. Boczkowsi, Pablo (2004). Digitizing the news. Innovation in online newspapers. Cambridge: MIT Press. ISBN: 978 02 62268 84 4
  3. Canavilhas, João (2022). “Inteligencia artificial aplicada al periodismo: traducción automática y recomendación de contenidos en el proyecto ‘A European Perspective’ (UER)”. Revista latina de comunicación social, n. 80. https://www.doi.org/10.4185/RLCS-2022-1534 · DOI
  4. Cavaliere, Paolo (2021). “From journalistic ethics to fact-checking practices: defining the standards of content governance in the fight against disinformation”. Journal of media law, v. 12, n. 2, pp. 133-165. https://doi.org/10.2139/ssrn.3769410 · doi:10.1080/17577632.2020.1869486
  5. Cazalens, Sylvie; Lamarre, Philippe; Leblay, Julien; Manolescu, Ioana; Tannier, Xavier (2018). “A content management perspective on fact-checking”. In: WWW’18 companion: the 2018 web conference companion, pp. 565-574. https://doi.org/10.1145/3184558.3188727 · DOI
  6. Cybenko, Anne K.; Cybenko, George (2018). „AI and fake news”. IEEE intelligent systems, n. 33. https://doi.org/10.1109/MIS.2018.2877280 · DOI
  7. Diakopoulos, Nicholas (2019). Automating the news: how algorithms are rewriting the media. Cambridge: Harvard University Press. ISBN: 978 0 674 23930 2 · DOI · Paper page
  8. Dörr, Konstantin-Nicholas (2016). “Mapping the field of algorithmic journalism”. Digital journalism, v. 4, n. 6, pp. 700-722. https://doi.org/10.1080/21670811.2015.1096748 · DOI · Paper page
  9. Ferrucci, Patrick; Perreault, Gregory (2021). “The liability of newness: journalism, innovation and the issue of core competencies”. Journalism studies, v. 22, n. 11, pp. 1436-1449. https://doi.org/10.1080/1461670X.2021.1916777 · DOI
  10. Flores-Vivar, Jesús-Miguel (2019). “Inteligencia artificial y periodismo: diluyendo el impacto de la desinformación y las noticias falsas a través de los bots”. Doxa comunicación, n. 29, pp. 197-212. https://doi.org/10.31921/doxacom.n29a10 · DOI
  11. Flores-Vivar, Jesús-Miguel (2020). “Datos masivos, algoritmización y nuevos medios frente a desinformación y fake news. Bots para minimizar el impacto en las organizaciones”. Comunicación y hombre, n. 16, pp. 101-114. https://doi.org/10.32466/eufv-cyh.2020.16.601.101-114 · doi:10.32466/eufv-cyh.2020.16.601.101-114
  12. Franganillo, Jorge (2022). “Contenido generado por inteligencia artificial: oportunidades y amenazas”. Anuario ThinkEPI, v. 16. https://doi.org/10.3145/thinkepi.2022.e16a24 · DOI
  13. García-Avilés, José-Alberto; Carvajal-Prieto, Miguel; Arias-Robles, Félix (2018). “Implantación de la innovación en los cibermedios españoles: análisis de las percepciones de los periodistas”. Revista latina de comunicación social, n. 73, pp. 369-384. https://doi.org/10.4185/RLCS-2018-1260 · doi:10.4185/rlcs-2018-1260
  14. García-Marín, David (2022). “Modelos algorítmicos y fact-checking automatizado. Revisión sistemática de la literatura”. Documentación de ciencias de la información, v. 45, n. 1, pp. 7-16. https://doi.org/10.5209/dcin.77472 · DOI
  15. García-Vivero, Gloria; López-García, Xosé (2021). “La verificación de datos en Europa. Análisis de 5 iniciativas europeas: Maldita.es, Newtral, Pagella Politica, Les Décodeurs y BBC Reality Check”. AdComunica, pp. 235-264. https://doi.org/10.6035/2174-0992.2021.21.12 · DOI
  16. Gómez-de-Ágreda, Ángel; Feijóo, Claudio; Salazar-García, Idoia-Ana (2021). “Una nueva taxonomía del uso de la imagen en la conformación interesada del relato digital. Deep fakes e inteligencia artificial”. Profesional de la información, v. 30, n. 2, e300216. https://doi.org/10.3145/epi.2021.mar.16 · DOI
  17. Graves, Lucas (2016). Deciding what’s true. The rise of political fact-checking in American journalism. New York: Columbia University Press. ISBN: 978 0 231175067 · DOI · Paper page
  18. Graves, Lucas (2017). “Anatomy of a fact check: objective practice and the contested epistemology of fact checking”. Communication, culture and critique, v. 10, n. 3, pp. 518-537. https://doi.org/10.1111/cccr.12163 · DOI · Paper page
  19. Graves, Lucas; Anderson, Christopher (2020). “Discipline and promote: building infrastructure and managing algorithms in a ‘structured journalism’ project by professional fact-checking groups”. New media & society, v. 22, n. 2, pp. 342-360. https://doi.org/10.1177/1461444819856916 · DOI
  20. Guallar, Javier; Codina, Lluís; Freixa, Pere; Pérez-Montoro, Mario (2020). “Desinformación, bulos, curación y verificación. Revisión de estudios en Iberoamérica 2017-2020”. Telos: Revista de estudios interdisciplinarios en ciencias sociales, v. 22, n. 3, pp. 595-613. https://doi.org/10.36390/telos223.09 · doi:10.36390/telos223.09
  21. Gupta, Ashish; Li, Han; Farnoush, Alireza; Jiang, Wenting (2022). “Understanding patterns of COVID infodemic: a systematic and pragmatic approach to curb fake news”. Journal of business research, v. 140, pp. 670-683. https://doi.org/10.1016/j.jbusres.2021.11.032 · DOI
  22. Hameleers, Michael; Van-der-Meer, Toni G. L. A. (2020). “Misinformation and polarization in a high-choice media environment: how effective are political fact-checkers?”. Communication research, v. 47, n. 2, pp. 227-250. https://doi.org/10.1177/0093650218819671 · DOI · Paper page
  23. Hassan, Naeemul; Adair, Bill; Hamilton, James T.; Li, Chengkai; Tremayne, Mark; Yang, Jun; Yu, Cong (2015). “The quest to automate fact-checking”. In: Proceedings of the 2015 computation + journalism symposium. Columbia University. http://cj2015.brown.columbia.edu/papers/automate-fact-checking.pdf
  24. Herrero, Esperanza; Herrera-Damas, Susana (2021). “El fact-checking hispanohablante: competencias, dificultades y propuestas de mejora desde la perspectiva de sus profesionales”. Profesional de la información, v. 30, n. 6, e300612. https://doi.org/10.3145/epi.2021.nov.12 · DOI
  25. Holman, Lynette; Perreault, Gregory (2023). “Diffusion of innovations in digital journalism: technology, roles, and gender in modern newsrooms”. Journalism, v. 24, n. 5, pp. 938-957. https://doi.org/10.1177/14648849211073441 · DOI
  26. Jamil, Sadia (2020). “Artificial intelligence and journalistic practice: the crossroads of obstacles and opportunities for the Pakistani journalists”. Journalism practice, v. 15, n. 10, pp. 1400-1422. https://doi.org/10.1080/17512786.2020.1788412 · DOI
  27. López-Pan, Fernando; Rodríguez-Rodríguez, Jorge-Miguel (2020). “El fact-checking en España. Plataformas, prácticas y rasgos distintivos”. Estudios sobre el mensaje periodístico, v. 26, n. 3, pp. 1045-1065. https://doi.org/10.5209/esmp.65246 · DOI
  28. Manfredi-Sánchez, Juan Luis; Ufarte-Ruiz, María-José (2020). “Inteligencia artificial y periodismo: una herramienta contra la desinformación”. Revista Cidob d’afers internacionals, n. 124, pp. 49-72. https://doi.org/10.24241/rcai.2020.124.1.49 · DOI
  29. Marconi, Francesco (2020). Newsmakers. Artificial intelligence and the future of journalism. New York: Columbia University Press. ISBN: 978 0 231 19137 1 · DOI
  30. Martínez-García, Luisa; Ferrer, Iliana (2023). “Fact-checking journalism: a palliative against the COVID-19 infodemic in Ibero-America”. Journalism & mass communication quarterly, v. 100, n. 2, pp. 264-285. https://doi.org/10.1177/10776990231164168 · DOI
  31. McIntyre, Karen Elizabeth; Lough, Kyser (2021). “Toward a clearer conceptualization and operationalization of solutions journalism”. Journalism, v. 22, n. 6, 1558-1573. https://doi.org/10.1177/1464884918820756 · DOI
  32. Miller, Jody; Glassner, Barry (2016). “The ‘inside’ and the ‘outside’: finding realities in interviews”. In: Silverman, David (ed.). Qualitative research. London: Sage. ISBN: 978 1 529 71297 1
  33. Moreno-Gil, Victoria; Ramon-Vegas, Xavier; Mauri-Ríos, Marcel (2022). “Bringing journalism back to its roots: examining fact-checking practices, methods, and challenges in the Mediterranean context”. Profesional de la información, v. 31, n. 2, e310215. https://doi.org/10.3145/epi.2022.mar.15 · DOI
  34. Moreno-Gil, Victoria; Ramon-Vegas, Xavier; Rodríguez-Martínez, Ruth (2021). “Fact-checking interventions as counteroffensives to disinformation growth: Standards, values, and practices in Latin America and Spain”. Media and communication, v. 9, n. 1, pp. 251-263. https://doi.org/10.17645/mac.v9i1.3443 · DOI
  35. Noain-Sánchez, Amaya (2022). “Addressing the impact of artificial intelligence on journalism: the perception of experts, journalists and academics”. Communication & society, v. 35, n. 3, pp. 105-121. https://doi.org/10.15581/003.35.3.105-121 · DOI · Paper page
  36. Oelrichs, Inga (2023). “Adoption of innovations in digital sports journalism: the use of Twitter by German sports journalists”. Communication & sport, v. 11, n. 2, pp. 288-312. https://doi.org/10.1177/2167479520961786 · doi:10.1177/2167479520961786
  37. Palomo, Bella; Sedano-Amundarain, Jon Ander (2018). “WhatsApp como herramienta de verificación de fake news. El caso de B de Bulo”. Revista latina de comunicación social, v. 73, pp. 1384-1397. https://doi.org/10.4185/RLCS-2018-1312 · doi:10.4185/rlcs-2018-1312
  38. Pasquetto, Irene V.; Jahani, Eaman; Atreja, Shubham; Baum, Matthew (2022). “Social debunking of misinformation on WhatsApp: the case for strong and in-group ties”. Proceedings of the ACM on human-computer interaction, v. 6. https://doi.org/10.1145/3512964 · doi:10.31219/osf.io/acvn3
  39. Peña-Fernández, Simón; Meso-Ayerdi, Koldobika; Larrondo-Ureta, Ainara; Díaz-Noci, Javier (2023). “Without journalists, there is no journalism: the social dimension of generative artificial intelligence in the media”. Profesional de la información, v. 32, n. 2. https://doi.org/10.3145/epi.2023.mar.27 · DOI
  40. Pérez-Dasilva, Jesús-Ángel; Meso-Ayerdi, Koldobika; Mendiguren-Galdospín, Terese (2020). “Fake news y coronavirus: detección de los principales actores y tendencias a través del análisis de las conversaciones en Twitter”. Profesional de la información, v. 29, n. 3. https://doi.org/10.3145/epi.2020.may.08 · DOI · Paper page
  41. Pérez-Seoane, Jesús; Corbacho-Valencia, Juan-Manuel; Dafonte-Gómez, Alberto (2023). “Análisis de las publicaciones con mayor repercusión en Facebook de los fact-checkers iberoamericanos en 2021”. Icono 14. Revista científica de comunicación y tecnologías emergentes, v. 21, n. 1. https://doi.org/10.7195/ri14.v21i1.1951 · DOI
  42. Ramon-Vegas, Xavier; Mauri-Ríos, Marcel; Rodríguez-Martínez, Ruth (2020). “Redes sociales y plataformas de fact-checking contra la desinformación sobre la COVID-19”. Hipertext.net, v. 21, pp. 79-92. https://doi.org/10.31009/hipertext.net.2020.i21.07 · doi:10.31009/hipertext.net.2020.i21.07
  43. Rodríguez-Pérez, Carlos (2020). “Una reflexión sobre la epistemología del fact-checking journalism: retos y dilemas”. Revista de comunicación, v. 19, n. 1, pp. 243-258. https://doi.org/10.26441/RC19.1-2020-A14 · DOI
  44. Rojas-Torrijos, José-Luis; Toural-Bran, Carlos (2019). “Periodismo deportivo automatizado. Estudio de caso de AnaFut, el bot desarrollado por El Confidencial para la escritura de crónicas de fútbol”. Doxa comunicación. Revista interdisciplinar de estudios de comunicación y ciencias sociales, n. 29, pp. 235-254. https://doi.org/10.31921/doxacom.n29a12 · DOI
  45. Russell, Stuart; Norvig, Peter (2022). Artificial intelligence: a modern approach. Fourth ed. London: Pearson Education. ISBN: 978 1 292 40117 1
  46. Salaverría, Ramón; Buslón, Nataly; López-Pan, Fernando; León, Bienvenido; López-Goñi, Ignacio; Erviti, María-Carmen (2020). “Desinformación en tiempos de pandemia: tipología de los bulos sobre la Covid-19”. Profesional de la información, v. 29, n. 3, e290315. https://doi.org/10.3145/epi.2020.may.15 · DOI · Paper page
  47. Sánchez-García, Pilar; Merayo-Álvarez, Noemí; Calvo-Barbero, Carla, Diez-Gracia, Alba (2023). “Spanish technological development of artificial intelligence applied to journalism: companies and tools for documentation, production and distribution of information”. Profesional de la información, v. 32, n. 2, e320208. https://doi.org/10.3145/epi.2023.mar.08 · DOI · Paper page
  48. Sánchez-Gonzales, Hada M.; Sánchez-González, María (2017). “Los bots como servicio de noticias y de conectividad emocional con las audiencias. El caso de Politibot”. Doxa comunicación. Revista interdisciplinar de estudios de comunicación y ciencias sociales, v. 25, pp. 63-84. https://doi.org/10.31921/doxacom.n25a3 · DOI
  49. Sánchez-González, María; Sánchez-Gonzales, Hada M.; Martínez-Gonzalo, Sergio (2022). “Inteligencia artificial en verificadores hispanos de la red IFCN: proyectos innovadores y percepción de expertos y profesionales”. Estudios sobre el mensaje periodístico, v. 28, n. 4, pp. 867-879. https://doi.org/10.5209/esmp.82735 · DOI
  50. Singer, Jane B. (2021). “Border patrol: the rise and role of fact-checkers and their challenge to journalists’ normative boundaries”. Journalism, v. 22, n. 8, pp. 1929-1946. https://doi.org/10.1177/1464884920933137 · DOI
  51. Soto-Sanfiel, María-Teresa; Ibiti, Adriana; Machado, Mabel; Marín-Ochoa, Beatriz-Elena; Mendoza-Michilot, María; Rosell-Arce, Claudio-Guillermo; Angulo-Brunet, Ariadna (2022). “In search of the Global South: assessing attitudes of Latin American journalists to artificial intelligence in journalism.” Journalism studies, n. 23, v. 10, pp. 1197-1224. https://doi.org/10.1080/1461670X.2022.2075786 · DOI · Paper page
  52. Sun, Mengru; Hu, Wencai; Wu, Yun (2022). “Public perceptions and attitudes towards the application of artificial intelligence in journalism: from a China-based survey.” Journalism practice. https://doi.org/10.1080/17512786.2022.2055621 · DOI · Paper page
  53. Tandoc Jr., Edson C.; Lim, Zheng Wei; Ling, Richard (2018). “Defining ‘fake news’. A typology of scholarly definitions”. Digital journalism, v. 6, n. 2, pp. 137-153. https://doi.org/10.1080/21670811.2017.1360143 · DOI · Paper page
  54. Thorne, James; Vlachos, Andreas (2018). “Automated fact checking: task formulations, methods and future directions”. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 3346-3359. https://aclanthology.org/C18-1283
  55. Tracy, Sarah J. (2020). Qualitative research methods. Collecting evidence, crafting analysis, communicating impact. Hoboken, New Jersey: Wiley Blackwell. ISBN: 978 1 405 19202 6
  56. Ufarte-Ruiz, María-José; Manfredi-Sánchez, Juan-Luis (2019). “Algoritmos y bots aplicados al periodismo. El caso de Narrativa Inteligencia Artificial: estructura, producción y calidad informativa”. Doxa comunicación. Revista interdisciplinar de estudios de comunicación y ciencias sociales, v. 29, pp. 213-233. https://doi.org/10.31921/doxacom.n29a11 · DOI
  57. Ufarte-Ruiz, María-José; Peralta-García, Lidia; Murcia-Verdú, Francisco-José (2018). “Fact checking: un nuevo desafío del periodismo”. El profesional de la información, v. 27, n. 4, pp. 733-741. https://doi.org/10.3145/epi.2018.jul.02 · DOI
  58. Van-Dalen, Arjen (2012). “The algorithms behind the headlines”. Journalism practice, v. 6, n. 5-6, pp. 648-658. https://doi.org/10.1080/17512786.2012.667268 · DOI · Paper page
  59. Vara-Miguel, Alfonso; Amoedo-Casais, Avelino; Moreno-Moreno, Elsa; Negredo-Bruna, Samuel; Kaufmann-Argueta, Jürg (2022). Digital News Report España 2022. Servicio de publicaciones de la Universidad de Navarra. https://doi.org/10.15581/019.2022 · doi:10.15581/019.2022
  60. Vizoso, Ángel; Vázquez-Herrero, Jorge (2019). “Fact-checking platforms in Spanish. Features, organisation and method”. Communication & society, v. 32, n. 1, pp. 127-142. https://doi.org/10.15581/003.32.37819 · doi:10.15581/003.32.37819
  61. Yousuf, Mohammad (2023). “Mediating the truth: Influences of routines on legacy news media fact-checking”. Journalism practice. https://doi.org/10.1080/17512786.2023.2169187 · DOI

Cited by

Metadata

Authors
Cuartielles, R., Ramon-Vegas, X., & Pont-Sorribes, C.
Affiliations
Universitat Pompeu Fabra. UPF Barcelona School of Management
Venue
El Profesional de la información
Published
2023-09-28
ISSN
  • 1699-2407
  • 1386-6710
Cited by
25
References
11

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.