CoChat
Sign Up
Cited by 17Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AISafety Systems Engineering in Autonomy

Mind The Gap: Designers and Standards on Algorithmic System Transparency for Users

Open access
Schor, Bianca G. S., Norval, Chris, Charlesworth, Ellen, Singh, Jatinder·2024·Proceedings of the CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3613904.3642531
Open this paper in CoChat — free accountOpen PDF

Abstract

Many call for algorithmic systems to be more transparent, yet it is often unclear for designers how to do so in practice. Standards are emerging that aim to support designers in building transparent systems, e.g by setting testable transparency levels, but their efficacy in this regard is not yet understood. In this paper, we use the ‘Standard for Transparency of Autonomous Systems’ (IEEE 7001) to explore designers’ understanding of algorithmic system transparency, and the degree to which their perspectives align with the standard’s recommendations. Our mixed-method study reveals participants consider transparency important, difficult to implement, and welcome support. However, despite IEEE 7001’s potential, many did not find its recommendations particularly appropriate. Given the importance and increased attention on transparency, and because standards like this purport to guide system design, our findings reveal the need for ‘bridging the gap’, through (i) raising designers’ awareness about the importance of algorithmic system transparency, alongside (ii) better engagement between stakeholders (i.e. standards bodies, designers, users). We further identify opportunities towards developing transparency best practices, as means to help drive more responsible systems going forward.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642531

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The integration of user-centred methods and design insights with governance, standards, and policymaking remains underdeveloped.

    “This lack of integration between HCI research and public policy is particularly salient in AI: while user-centred methods and design insights have clear potential to support responsible development and deployment, their integration with governance, standards, and policymaking remains underdeveloped [75, 145].”

    From “It’s Just a Wild, Wild West”: Harnessing Public Procurement as an AI Governance Mechanism (2026) · cites this paper as [145]

    Section 6.1.4 identifies a gap between transparency standards’ recommendations and designers’ approaches, and calls for closer engagement among standards organisations, designers, and users. That supports the claim about underdeveloped integration with standards, but the paper does not establish a comparable gap between user-centred design and governance or policymaking more broadly. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Public procurement remains an underexplored field in HCI.

    “Despite the fact that AI governance and public policy are increasingly recognised as important ways forward in the CHI community [e.g. 65, 76, 100, 103, 145, 164, 167, 177], public procurement remains an underexplored field in HCI [75, 145, 172].”

    From “It’s Just a Wild, Wild West”: Harnessing Public Procurement as an AI Governance Mechanism (2026) · cites this paper as [145]

    The Introduction mentions procurement specialists as people who can use standards to set specifications, but it does not discuss public procurement as an HCI research field. A search for public procurement and related terms found no passage saying it is underexplored in HCI. Page numbers were unavailable in our copy. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 49 · Cited by 17

Research works printed but not linked: 47

  1. Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y Lim, and Mohan Kankan- halli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 ACM Human and Computer Interaction (CHI) conference on human factors in computing systems (CHI'18). Association for Computing Machinery, New York, NY, USA, 1-18.
  2. Rachel Adams. 2020. Transparency: new trajectories in law. Routledge.
  3. The Alan Turing Institute AI Standards Hub. 2023. Towards transpar- ent and explainable AI: Workshop to inform ISO/IEC standards develop- ment. https://aistandardshub.org/events/towards-transparent-and-explainable- ai-workshop-to-inform-iso-iec-standards-development/
  4. Robert H Allen and Ram D Sriram. 2000. The Role of standards in innovation. Technological Forecasting and Social Change 64, 2-3 (2000), 171-181.
  5. Mike Ananny and Kate Crawford. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. new media & society 20, 3 (2018), 973-989.
  6. Carolyn Ball. 2009. What is transparency? Public Integrity 11, 4 (2009), 293-308.
  7. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yun- han Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. 2020. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 648- 657. https://doi.org/10.1145/3351095.3375624 · DOI · Paper page
  8. Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the people? Opportunities and challenges for participatory AI. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Arlington, VA, USA) (EAAMO '22). Association for Computing Ma- chinery, New York, NY, USA, 8 pages. https://doi.org/10.1145/3551624.3555290 · DOI · Paper page
  9. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.
  10. Raja Chatila and John C Havens. 2019. The IEEE global initiative on ethics of autonomous and intelligent systems. In Robotics and well-being. Springer, 11-16.
  11. Eric Corbett and Emily Denton. 2023. Interrogating the T in FAccT. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 1624-1634.
  12. Upol Ehsan, Philipp Wintersberger, Q. Vera Liao, Martina Mara, Marc Streit, Sandra Wachter, Andreas Riener, and Mark O. Riedl. 2021. Operationalizing human-centered perspectives in explainable AI. In Extended Abstracts of the 2021 ACM Human-Computer Interation (CHI) Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA '21). Association for Computing Machinery, New York, NY, USA, 6 pages. https://doi.org/10.1145/3411763.3441342 · DOI
  13. Upol Ehsan, Philipp Wintersberger, Q Vera Liao, Elizabeth Anne Watkins, Carina Manger, Hal Daumé III, Andreas Riener, and Mark O Riedl. 2022. Human-Centered Explainable AI (HCXAI): beyond opening the black-box of AI. In CHI conference on human factors in computing systems extended abstracts. 1-7.
  14. Upol Ehsan, Philipp Wintersberger, Elizabeth A Watkins, Carina Manger, Gon- zalo Ramos, Justin D. Weisz, Hal Daumé Iii, Andreas Riener, and Mark O Riedl. 2023. Human-centered explainable AI (HCXAI): coming of age. In Extended Abstracts of the 2023 ACM Human-Computer Interation (CHI) Con- ference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA '23). Association for Computing Machinery, New York, NY, USA, 7 pages. https://doi.org/10.1145/3544549.3573832 · DOI
  15. Heike Felzmann, Eduard Fosch-Villaronga, Christoph Lutz, and Aurelia Tamò- Larrieux. 2020. Towards transparency by design for artifcial intelligence. Science and Engineering Ethics 26, 6 (2020), 3333-3361.
  16. IEEE. 2021. IEEE 7000™-2021. Integrating ethical and functional requirements to mitigate risk and increase innovation in systems engineering design and development. https://engagestandards.ieee.org/ieee-7000-2021-for-systems- design-ethical-concerns.html
  17. IEEE. 2021. P7001/D3 Draft Standard for Transparency of Autonomous Sys- tems. https://www.techstreet.com/ieee/standards/ieee-p7001?gateway_code= ieee&vendor_id=6929&product_id=2188891
  18. Nanna Inie and Peter Dalsgaard. 2017. How interaction designers use tools to capture, manage, and collaborate on ideas. In Proceedings of the 2017 ACM Human-COomputer Interaction (CHI) Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 2668-2675. https://doi.org/10. 1145/3027063.3053210 · DOI
  19. Milena Janic, Jan Pieter Wijbenga, and Thijs Veugen. 2013. Transparency en- hancing tools (TETs): an overview. In 2013 Third Workshop on Socio-Technical Aspects in Security and Trust. IEEE, 18-25.
  20. Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting interpretability: understand- ing data scientists' use of interpretability tools for machine learning. In Proceed- ings of the 2020 ACM Human-computer Interaction (CHI) conference on human factors in computing systems (CHI'2020). Association for Computing Machinery, New York, NY, USA, 1-14.
  21. Ansgar Koene, Chris Clifton, Yohko Hatada, Helena Webb, and Rashida Richard- son. 2019. A governance framework for algorithmic accountability and trans- parency. (2019). https://doi.org/10.2861/59990 · DOI
  22. Joshua Alexander Kroll, Joanna Huey, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu. 2017. Accountable algorithms. University of Pennsylvania Law Review 165 (2017), 633-705. https://scholarship. law.upenn.edu/penn_law_review/vol165/iss3/3/
  23. Stefan Larsson and Fredrik Heintz. 2020. Transparency in artifcial intelligence. Internet Policy Review 9, 2 (2020).
  24. David Lehr and Paul Ohm. 2017. Playing with the data: what legal scholars should learn about machine learning. UC Davis Law Review 51 (2017).
  25. Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: inform- ing design practices for explainable AI user experiences. In Proceedings of the 2020 ACM Human-Computer Interaction (CHI) Conference on Human Factors in Computing Systems (CHI'20). Association for Computing Machinery, New York, NY, USA, 1-15.
  26. Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-designing checklists to understand organizational challenges and opportunities around fairness in AI. In Proceedings of the 2020 ACM Human- Computer Interaction (CHI) conference on human factors in computing systems (CHI'20). Association for Computing Machinery, New York, NY, USA, 1-14.
  27. Henrik Mucha, Dennis Mevißen, Sebastian Robert, Ricarda Jacobi, Kirsten Meyer, Winfried Heusler, and Daniel Arztmann. 2020. Co-Design futures for AI and space: a workbook sprint. In Extended Abstracts of the 2020 ACM Human-Computer Interation (CHI) Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1-8. https://doi.org/10.1145/3334480.3375203 · DOI
  28. Chris Norval, Kristin Cornelius, Jennifer Cobbe, and Jatinder Singh. 2022. Dis- closure by design: designing information disclosures to support meaningful transparency and accountability. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22). Association for Computing Machinery, New York, NY, USA, 679-690. https://doi.org/10.1145/3531146.3533133 · DOI
  29. Chris Norval and Jatinder Singh. 2023. A Room With an Overview: Towards Meaningful Transparency for the Consumer Internet of Things. IEEE Internet of Things Journal (2023), 1-1. https://doi.org/10.1109/JIOT.2023.3318369 · DOI
  30. Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where responsible AI meets reality: practitioner perspectives on enablers for shifting organizational practices. Proceedings of the 2021 ACM conference on Human-Computer Interaction 5, CSCW'21, 1-23.
  31. Gonzalo Ramos, Jina Suh, Soroush Ghorashi, Christopher Meek, Richard Banks, Saleema Amershi, Rebecca Fiebrink, Alison Smith-Renner, and Gagan Bansal. 2019. Emerging perspectives in human-centered machine learning. In Ex- tended Abstracts of the 2019 ACM Human-Computer Interaction (CHI) Confer- ence on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1-8. https: //doi.org/10.1145/3290607.3299014 · DOI
  32. Mary Beth Rosson, Susanne Maass, and Wendy A. Kellogg. 1986. Designing for designers: an analysis of design practice in the real world. SIGCHI Bulletin 17, SI (May 1986), 137-142. https://doi.org/10.1145/30851.30873 · DOI
  33. Daniel Schif, Bogdana Rakova, Aladdin Ayesh, Anat Fanti, and Michael Lennon. 2020. Principles to practices for responsible AI: closing the gap. arXiv preprint arXiv:2006.04707 (2020). · arXiv
  34. Kacper Sokol and Peter Flach. 2020. Explainability fact sheets: a framework for systematic assessment of explainable approaches. In Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency (FAccT'20). Association for Computing Machinery, New York, NY, USA, 56-67.
  35. Marc Steen, Menno Manschot, and Nicole De Koning. 2011. Benefts of co-design in service design projects. International Journal of Design 5, 2 (2011).
  36. Cynthia Stohl, Michael Stohl, and Paul M Leonardi. 2016. Managing opacity: information visibility and the paradox of transparency in the digital age. Inter- national Journal of Communication 10 (2016). https://ijoc.org/index.php/ijoc/ article/view/4466/1530
  37. European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Ofcial Journal of the European Union L119 (4 May 2016), 1-88.
  38. Andrew Walter. 2011. Governing fnance: East Asia's adoption of international standards. Cornell University Press.
  39. Qiaosi Wang, Michael Madaio, Shaun Kane, Shivani Kapania, Michael Terry, and Lauren Wilcox. 2023. Designing responsible AI: adaptations of UX practice to meet responsible AI challenges. In Proceedings of the 2023 ACM Human-Computer Interaction (CHI) Conference on Human Factors in Computing Systems (CHI'23). Association for Computing Machinery, New York, NY, USA, 1-16.
  40. Adrian Weller. 2019. Transparency: motivations and challenges. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, 23-40.
  41. Maranke Wieringa. 2020. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency (FAT* '20). Association for Computing Machinery, New York, NY, USA, 1-18. https: //doi.org/10.1145/3351095.3372833 · DOI · Paper page
  42. Rebecca Williams, Richard Cloete, Jennifer Cobbe, Caitlin Cottrill, Peter Edwards, Milan Markovic, Iman Naja, Frances Ryan, Jatinder Singh, and Wei Pang. 2022. From transparency to accountability of intelligent systems: moving beyond aspirations. Data & Policy 4 (2022), e7. https://doi.org/10.1017/dap.2021.37 · DOI
  43. Alan FT Winfeld, Serena Booth, Louise A Dennis, Takashi Egawa, Helen Hastie, Naomi Jacobs, Roderick I Muttram, Joanna I Olszewska, Fahimeh Rajabiyazdi, Andreas Theodorou, et al. 2021. IEEE P7001: a proposed standard on transparency. Frontiers in Robotics and AI (2021), 225.
  44. Ning Xie, Gabrielle Ras, Marcel van Gerven, and Derek Doran. 2020. Explainable deep learning: A feld guide for the uninitiated. arXiv preprint arXiv:2004.14545 (2020). · arXiv
  45. Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fer- nanda Viégas. 2023. Investigating Hhw practitioners use human-AI guidelines: a case study on the People+ AI guidebook. In Proceedings of the 2023 ACM Human- Computer Interaction (CHI) Conference on Human Factors in Computing Systems (CHI'23). Association for Computing Machinery, New York, NY, USA, 1-13.
  46. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Efect of confdence and explanation on accuracy and trust calibration in AI-assisted decision mak- ing. In Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency (FAccT'20). Association for Computing Machinery, New York, NY, USA, 295-305.
  47. Douglas Zytko, Pamela J. Wisniewski, Shion Guha, Eric P. S. Baumer, and Min Kyung Lee. 2022. Participatory design of AI systems: opportunities and chal- lenges across diverse users, relationships, and application domains. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA '22). Association for Computing Machinery, New York, NY, USA, 4 pages. https://doi.org/10.1145/3491101.3516506 · DOI · Paper page

Authors
Schor, B. G. S., Norval, C., Charlesworth, E., & Singh, J.
Affiliations
  • University of Cambridge (GB)
  • University of Aberdeen (GB)
  • Durham University (GB)
ORCID
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-16
DOI
10.1145/3613904.3642531
OpenAlex
W4396831964
Cited by
17
References
49
Citations by year
2026: 7 · 2025: 9 · 2024: 1
FWCI
4.91
Type
conference paper
Language
en
Open access
Licence
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • Durham Research Online (Durham University) (publishedVersion, other-oa)
  • Aberdeen University Research Archive (Aberdeen University) (submittedVersion)
Funders
Trinity college, Cambridge
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Keywords
algorithmic transparency, algorithmic systems, transparency standards, designer perspective, autonomous systems, responsible design, stakeholder engagement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review