The DOI matches its Crossref record; 62 of the 64 references it cites were found in a registry or an index, and 2 are not checked yet; none of the 20 sources its claims cite is checked for support yet.
Interactive Model Cards: A Human-Centered Approach to Model Documentation
Open accessAbstract
Deep learning models for natural language processing (NLP) are increasingly adopted and deployed by analysts without formal training in NLP or machine learning (ML). However, the documentation intended to convey the model’s details and appropriate use is tailored primarily to individuals with ML or NLP expertise. To address this gap, we conduct a design inquiry into interactive model cards, which augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves. Our investigation consists of an initial conceptual study with experts in ML, NLP, and AI Ethics, followed by a separate evaluative study with non-expert analysts who use ML models in their work. Using a semi-structured interview format coupled with a think-aloud protocol, we collected feedback from a total of 30 participants who engaged with different versions of standard and interactive model cards. Through a thematic analysis of the collected data, we identified several conceptual dimensions that summarize the strengths and limitations of standard and interactive model cards, including: stakeholders; design; guidance; understandability & interpretability; sensemaking & skepticism; and trust & safety. Our findings demonstrate the importance of carefully considered design and interactivity for orienting and supporting non-expert analysts using deep learning models, along with a need for consideration of broader sociotechnical contexts and organizational dynamics. We have also identified design elements, such as language, visual cues, and warnings, among others, that support interactivity and make non-interactive content accessible. We summarize our findings as design guidelines and discuss their implications for a human-centered approach towards AI/ML documentation.
Checked 2026-10-06 · 2 references not checked yet
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper designs and evaluates interactive model cards (IMCs) that augment static model cards with interactivity and guidance for non-expert NLP/ML analysts. Across two qualitative studies with 30 participants, IMCs were generally favored over standard model cards for understandability and sharing, though ethics and safety content remained hard to make actionable.
Key findings
- In Study 2, nearly all 20 non-expert analysts (n=18) found the IMC easier to read and understand; n=15 said they would use it in daily work, while n=2 said it was context-dependent, and the IMC was overall favored over the standard model card (SMC).
- With the SMC alone, almost half of Study 2 participants could not correctly interpret that the model performed sentiment analysis: four thought it was a general text classification model, two thought DistilBERT and its fine-tuned version were both sentiment analysis models, one did not answer, and one thought the SST-2 dataset was the model; even three who correctly described it were not confident.
- IMC design elements helped engagement: layout (n=13), language (n=6), ability to add examples (n=15), and visual/interactive elements (n=16); n=16 also suggested improvements, and n=4 reported information overload.
- Study 1 experts surfaced conceptual dimensions including stakeholders, design, guidance, trust and safety, and sensemaking and skepticism, leading to design guidelines DG1–DG4 for interactive model cards.
Methods
- Two-study design inquiry: Study 1 was a conceptual study with 10 experts in ML/AI, NLP, ethics, and auditability using paper-prototype probes of an SMC, Robustness Gym report (RGR), and IMC; Study 2 was an evaluative study with 20 non-expert analysts using a functional IMC prototype compared with a reference SMC.
- Both studies used semi-structured interviews with a think-aloud protocol; Study 1 recorded approximately 11 hours 15 minutes of video, and Study 2 recorded approximately 23 hours 30 minutes; analysis used iterative thematic analysis with grounded theory, with Study 1 themes seeding Study 2 analysis.
- Study 2 recruitment used the User Interviews platform: 176 responses, 117 excluded (89 never used ML/NLP models; 28 had graduate or undergraduate training in ML, statistics, or computer science), leaving 59 candidates split into Type I (n=35, no or infrequent programming) and Type II (n=24, daily or weekly programming), from which 20 participants were recruited evenly; the reference model was DistilBERT fine-tuned on SST-2 for sentiment analysis.
Limitations
- Stated: design choices and the choice of sentiment analysis task likely influenced findings; specific IMC design elements may be refined in future work, though the broader design guidelines and conceptual dimensions are expected to generalize; creating and learning to use IMCs is a heavier burden than text-based model cards.
- Stated/future: future work should explore strategies for reducing and redistributing the work of model documentation.
- Evident from the text: the qualitative samples are small (10 experts and 20 analysts) and not statistically representative; Study 2 found no large response differences between analyst groups; ethics and safety remained difficult, with only four participants saying ethical templates could stimulate discussion and two pushing back against more actionable ethics/safety content.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
63 Own result · 32 Own interpretation · 9 Not checked yet · 3 Own method · 2 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, almost half of the participants were not able to provide the correct interpretation of what the model did using the SMC alone.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with participants echoing the concerns that were previously articulated by the experts in Section 3.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Supporting interpretability is complex but nearly all (n=7) participants indicated that examples could provide some useful guidance.”
hedgedn = 7- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A few participants (n=4) expressed that text, visuals, and interaction options led to information overload.”
plaindirectn = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The reference model card (Section 3.1), here acting as the SMC, and the IMC each had features that were preferred by both types of analysts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Compared to the prior study, non-expert analysts provided greater depth towards the themes of stakeholders ; design considerations ; and sensemaking & skepticism in particular.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“participants indicated that interactivity added substantially to the SCM because it allowed participants to 'play with the data and see the results [in a way that] that feels super intuitive instead of explaining everything' [A1-09].”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One of the most salient implications of the IMC design was that it helped participants accurately understand what the model was and what it did.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants indicated that various stakeholders would need help making sense of the information in the model card.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even three participants who did correctly describe the model were not confident in their interpretation”
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 24
- Probably not a substantial claim. 35
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, the addition of guiding elements, suggested in the previous study (Table 2), were well received and did help participants understand and interpret the model card information.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our conceptual study emphasized the importance of information design within both standard and interactive model cards.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants indicated that for these issues and overall, greater attention to information, visual, and interaction design would improve trust and safety.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Several (n=4) participants expressed that it could be beneficial to add sensitive populations to the disaggregated model performance, but they suggested caution since the definition of these groups was culturally and geographically contextual.”
hedgedn = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants suggested that integrating the model's code and files, present in the SMC, would be valuable for the IMC.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Warnings, prompts, and summaries are all examples of guidance that participants suggested could help guide analysts in making sense of model cards.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When asked, nearly all participants (n=15) indicated that the IMC was something they would use in their daily work, while some (n=2) indicated it was context-dependent.”
hedgedn = 15- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“one participant initially thought the model performed well for a particular subpopulation, then saw the small sample size and revised their interpretation.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the interactivity would enable people to 'elicit more questions' [E04] that allowed them to form and test their mental models about the data and better 'see the things that model lacks' [E05].”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants articulated that several components of the IMC helped them engage with the model and data more effectively by clarifying and prioritizing information, particularly: layout of information (n=13); language (n=6); ability to add examples (n=15); and visual and interactive elements (n=16).”
plaindirectn = 16- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“experts saw the opportunity to use interaction as a way to help non-expert analysts develop their conceptual understanding of the NLP model.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Three participants voiced the importance of being able to contest the model's results, and wanted to see this ability expanded beyond adding sentences (Section 4.1) to the model's training and testing data or uploading their own dataset.”
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Increasingly, such statements are seen as a 'cop-out because it just kind of says 'there is some bias somewhere' which is not entirely helpful.' [E06]”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nearly all (n=16) participants also made suggestions for improvements to the IMC design. These included changes to elements of the interface that were confusing (n=5), the layout (n=6), and types of guidance (n=1) provided.”
plainn = 16- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“they emphasized the need to prominently display the total size of these subpopulations (and others) to support valid interpretation.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We found that our approach to guidance, shareability, and traceability begins to address these needs, and we surfaced additional relevant implications for design.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
- Rests on an assumption: The DistilBERT SST-2 sentiment analysis task is representative of NLP model development regimens for other tasks.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All participants expressed doubt that a non-expert analyst would understand the model card or know how to act on the information it contained.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, a sense of focus/overwhelm varied among participants, and one participant articulated that the IMC contained 'too much information and is very cluttered' [E04].”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two participants suggested stating explicitly whether bias assessments had been conducted, and one suggested raising a warning if such bias analyses are not run.”
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Four participants stated that a summary of the model's purpose (i.e., 'this is a sentiment analysis model') was needed and that it would improve the readability of the model card.”
plaindirectn = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, having an effective default example to prompt end-users was also important [E04].”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the IMC included templates to examine ethically challenging examples (Section 4.1), only four participants expressed that these features could stimulate discussion.”
plainn = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While the IMC also provided template sentences to promote ethical thinking, participants largely favored their own examples.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, three participants expressed that IMCs, if they could be easy to produce, would lower their existing model documentation burden.”
hedgedconditionaln = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, our research supports prior findings that these concepts are challenging to integrate into decision-making [48, 60], and we identified some cynicism toward ethics statements, as well as assertions that ethical questions require contextual, situated examination.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Model cards are intended to surface ethical and safety implications, a topic especially pressing for deep learning models.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Others felt that topics of safety and ethics were too abstract, with one participant stating that they 'wouldn't worry too much about that to be honest, I guess it depends on the type of analysis you're doing.' [A2-05].”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, the reference model card contained information and actions to obtain the model code and launch it - something that was absent in the IMC.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unexpectedly, two participants pushed back against the inclusion of more actionable ethical and safety content, articulating concerns that too much guidance could be detrimental and provide a false sense of security.”
plaindirectn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One participant expressed doubt that any model documentation could really give an adequate picture of ethical and safety risks, emphasizing that ' 'this [model card] would not be my only data point or else, you know, that's a very quick way to lose your job' [A1-04].”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, to be sure that this content is effectively understood by a wide audience, the hierarchy and design of this content is equally important.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: Prior work related to SMCs has focused primarily on the categories and depth of information that model documentation should include.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Video transcripts were automatically generated, including speaker identification, and were verified for accuracy by the authors.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This depth allowed us to expand on the subthemes from Table 2 and broaden our conceptions of how model cards might be used.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All participants had exposure to documentation for ML/AI models in some form, but only six were familiar with model cards and two with RGRs.”
plainn = 10- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Robustness Gym helps model developers report on model behavior through the generation of robustness reports (RGRs) that summarize model performance on various data slices [23].”
What the sources say
- Not checked yetcites [23]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Clear guidance can support non-experts in understanding and knowing how to act on the information.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additional guidance will be necessary to support interpretability of the outputs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: SMCs have been primarily tailored toward developers and machine learning experts.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We argue that this productive skepticism would make ideas of intended and unintended uses more intuitive than simply being told what they are.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
- Rests on an assumption: Not all readers of a model card will want to interact with it.
None of the sources cited for this claim have been checked yet.
In the paper
“Prior work has shown that non-expert analysts benefit from interacting with machine learning models and their data [1, 19, 59].”
What the sources say
- Not checked yetcites [1]
- Not checked yetcites [19] source
- Not checked yetcites [59]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“As many researchers have articulated, safe and effective deployment of AI/ML systems in practice requires a robust understanding of different stakeholders' needs and goals for explainability [8, 32]; support for traceability and auditing [71]; and strategies to address concerns within existing organizational structures and dynamics [8, 31, 32, 57, 71].”
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Importantly, we found that scaffolding these interactions to support sensemaking requires clear and actionable guidance.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“1) Interactive model cards are a bridge to further analysis, not a substitute.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also recognize that creating and learning how to use IMCs is a heavier burden than for text-based model cards.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The recruited participants are representative of the future data workforce.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For model cards to be adopted and impact organizational processes, they must support knowledge sharing and negotiation across stakeholders with diverse backgrounds and perspectives.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We argue that with thoughtfully designed interaction, model cards can be a first step for data workers to explore how trustworthy a model is and whether it might meet their needs, serving as a guide to deeper interrogation when desired.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our findings indicate, unsurprisingly, that model documentation can spark ethical thinking, but will never substitute for it.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Model cards are intended to surface ethical and safety implications, a topic especially pressing for deep learning models.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Design choices and the choice of sentiment analysis tasks (Section 3.1) likely influenced our findings.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For individual stakeholders, the most effective form for interactivity in model cards will be situated and contextual.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Importantly, our analysis reinforces that considerations of ethics are personal and contextual, so they are unlikely to be addressed with a single approach.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Model cards are intended to surface ethical and safety implications, a topic especially pressing for deep learning models.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Their feedback validates many of our design choices, but also offers avenues for further improvements.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we believe that the broader design guidelines (Section 3.4) and conceptual dimensions that we surfaced (Tables 2, 4, and 5) are robust and will generalize to other tasks and contexts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: The recruited participants are representative of the future data workforce.
The paper relies on this without support.
Model cards and similar documentation rely on the developer to accurately and clearly report on the model and its performance.
“However, these model cards and similar forms of documentation rely on the developer to accurately and clearly report on the model and its performance.”
The paper relies on this without support.
Many important details, such as unintended uses or disaggregated model performance, are omitted from model cards.
“Often, this process is labor-intensive and many important details, such as unintended uses or disaggregated model performance, are omitted.”
The paper relies on this without support.
The DistilBERT SST-2 sentiment analysis task is representative of NLP model development regimens for other tasks.
“We chose to focus on a sentiment analysis task because it is widely known and easily understood, while still being representative of NLP model development regimens for other tasks.”
The paper relies on this without support.
The design probes (SMC, RGR, IMC) adequately represent the respective documentation types.
“We developed design probes [70] of an SMC, RGR, and IMC. For the SMC, we used the all the text from the reference model card, with some exceptions, laid out according to the specifications in Mitchell et al. [44].”
The paper relies on this without support.
The model card format is text-based and concise, making it both broadly accessible and applicable across model types.
“The format they propose is text-based and concise, making it both broadly accessible and applicable across model types.”
The paper relies on this without support.
Prior work related to SMCs has focused primarily on the categories and depth of information that model documentation should include.
“Prior work related to SMCs has focused primarily on the categories and depth of information that model documentation should include.”
The paper relies on this without support.
SMCs have been primarily tailored toward developers and machine learning experts.
“SMCs have been primarily tailored toward developers and machine learning experts”
The paper relies on this without support.
Not all readers of a model card will want to interact with it.
“Not all readers of a model card will want to interact with it.”
The paper relies on this without support.
Technology that enables low or no code data analysis is lowering barriers to developing and deploying deep learning models.
“Technology that enables low or no code data analysis is lowering barriers to developing and deploying deep learning models.”
The paper relies on this without support.
The data workers taking on AI/ML development do not have formal education in the theoretical and technical underpinnings of these systems.
“Increasingly, the data workers taking on AI/ML development do not have formal education in the theoretical and technical underpinnings of these systems.”
The paper relies on this without support.
The recruited participants are representative of the future data workforce.
“We recruited and explored the perspectives of people who we believe are representative of the future data workforce.”
The paper relies on this without support.
Model cards are intended to surface ethical and safety implications, a topic especially pressing for deep learning models.
“Model cards are intended to surface, amongother things, ethical and safety implications - a topic that is especially pressing for deep learning models.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Model cards represent the field's standard for disclosing and documenting key facts about the architecture, training and evaluation of the model [14, 40].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [14]
Citations
Cites
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826
Harris, C., Millman, K., van der Walt, S., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N., et al. (2020) · RePEc: Research Papers in Economics · Cited by 17,844
Data Structures for Statistical Computing in Python
McKinney, W. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-00a · Cited by 12,930
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Research works printed but not linked: 14
- Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Mag. 35(2014), 105–120.
- Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”.
- Berkeley J. Dietvorst, Joseph P. Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err.Journal of experimental psychology. General 144 1 (2015), 114–26.
- Alexandra Giannopoulou. 2020. Algorithmic Systems: the Consent is in the Detail?Internet Policy Review 9, 1 (2020).
- Karan Goel, Nazneen Rajani, Jesse Vig, Samson Tan, Jason Wu, Stephan Zheng, Caiming Xiong, Mohit Bansal, and Christopher Ré. 2021. Robustness Gym: Unifying the NLP Evaluation Landscape. arXiv preprint arXiv:2101.04840(2021). · arXiv
- Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A Survey of Methods for Explaining Black Box Models. ACM computing surveys (CSUR) 51, 5 (2018), 1–42.
- Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, 2019. WeBuildAI: Participatory Framework for Algorithmic Governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–35.
- Andreas Madsen Siva Reddy and Sarath Chandar. 2021. Post-hoc Interpretability for Neural NLP: A Survey. arXiv preprint arXiv:2108.04840(2021). · arXiv
- Christoph Molnar, Giuseppe Casalicchio, and Bernd Bischl. 2020. Interpretable Machine Learning–A Brief History, State-of-the-Art and Challenges. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 417–431.
- Michael Muller. 2014. Curiosity, Creativity, and Surprise as Analytic Tools: Grounded Theory Method. In Ways of Knowing in HCI. Springer, 25–48.
- Radim Rehurek and Petr Sojka. 2011. Gensim–python framework for vector space modelling. NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic 3, 2(2011).
- Salome Viljoen. 2021. A Relational Theory of Data Governance. The Yale Law Journal 131, 573 (2021).
- Meredith Whittaker, Kate Crawford, Roel Dobbe, Genevieve Fried, Elizabeth Kaziunas, Varoon Mathur, Sarah Mysers West, Rashida Richardson, Jason Schultz, and Oscar Schwartz. 2018. AI Now Report 2018. AI Now Institute at New York University New York.
- Jilei Yang, Diana Negoescu, and Parvez Ahammad. 2021. Intellige: A User-Facing Model Explainer for Narrative Explanations. arXiv preprint arXiv:2105.12941(2021). · arXiv
Cited by
Sorted by citations
Towards conversational diagnostic artificial intelligence
Tu, T., Schaekermann, M., Palepu, A., Saab, K., Freyberg, J., Tanno, R., Wang, A., Li, B., Amin, M., Cheng, Y., et al. (2025) · Nature · doi:10.1038/s41586-025-08866-7 · Cited by 366
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“Transparency can be achieved through data or model cards, documentation of the data and who is represented within it, as well as how the model was made, assumptions that were made, how the model behaves in relation to diverse groups of people, and how well the model performs in relation to those groups [45, 55, 68, 10…”
BackgroundAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“, interactive model cards [183] and information about the initial training dataset [184], [185], to help downstream developers adapt the model appropriately.”
BackgroundUnderstanding accountability in algorithmic supply chains
Cobbe, J., Veale, M., & Singh, J. P. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594073 · Cited by 141
“or using a system, such as for datasheets [39] or data cards [86] to describe datasets, or model cards [29, 75] and factsheets [7] to describe model specification and capabilities.”
BackgroundMethodLiesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111
“Interactive model cards [13] offer a promising step towards a humancentered approach to documentation.”
BackgroundLiao, Q. V., Subramonyam, H., Wang, J., & Vaughan, J. W. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580652 · Cited by 120
“Their struggles were not due to a lack of ML expertise [32], as participants had little difficulty comprehending the documentation (though this may not generalize to designers with little knowledge about AI).”
BackgroundNavigating LLM ethics: advancements, challenges, and future directions
Jiao, J., Afroogh, S., Xu, Y., & Phillips, C. (2025) · AI and Ethics · doi:10.1007/s43681-025-00814-5 · Cited by 171
“These approaches aim to protect user data while maintaining utility and accuracy of LLMs. Research directions increasingly focus on integrating privacy-preserving techniques directly into architecture and training processes of LLMs [68], [69], seeking balance between advanced language processing…”
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90
“Model cards are deployed across Hugging Face’s platform and have evolved to be interactive [26].”
BackgroundRethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 123
Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 65
“Model development An iterative process of deriving, testing, and building multiple models until a model fitting the desired criteria is developed (Arnold et al., 2019; Crisan et al., 2022; Mitchell et al., 2019)”
InfluentialZeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
Cabrera, Á. A., Fu, E., Bertucci, D., Holstein, K., Talwalkar, A., Hong, J. I., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581268 · Cited by 51
“Extensions to these reporting methods, namely Interactive Model Cards [16], have aimed to improve their usability by making them more expressive and interactive.”
MethodCertification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
Scharowski, N., Benk, M., Kühne, S. J., Wettstein, L., & Brühlmann, F. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593994 · Cited by 61
“Various methods have been proposed to increase AI systems’ transparency and, thereby auditability, such as through the use of model documentation or information about datasets [14, 21].”
BackgroundMethodThe Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
What Did My AI Learn? How Data Scientists Make Sense of Model Behavior
Cabrera, Á. A., Ribeiro, M. T., Lee, B., DeLine, R. A., Perer, A., & Drucker, S. M. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3542921 · Cited by 48
“[49] arXiv Spotlight [23] CVPR Barlow [77] ICDE Slice Finder [21] VIS FairVis [14] CHI ModelTracker [2] VIS Squares [67] N/A Facets [64] IUI AnchorVis [19] HILDA MLCube [42] CSCW Deblinder [13] ACL Errudite [89] ICLR Domino [24] VIS HypoML [83] ASE DeepRoad [92] ICSE DeepTest [79] ICSE Structure-Invariant Testing [35]…”
MethodLarge language models and political science
Linegar, M., Kocielnik, R., & Alvarez, R. M. (2023) · Frontiers in Political Science · doi:10.3389/fpos.2023.1257092 · Cited by 57
“Recent initiatives such as “Interactive Model Cards” ( Crisan et al., 2022), “AutoBiasTest” (Kocielnik et al., 2023b), and model written evaluations focus on interactive tools, that support live exploration of model capabilities and limitations using generated datasets or human-in-the-loop…”
Berman, G., Goyal, N., & Madaio, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642398 · Cited by 51
How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study
Gu, K., Grunde-McLaughlin, M., McNutt, A., Heer, J., & Althoff, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641891 · Cited by 40
Moore, S., Liao, Q. V., & Subramonyam, H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581242 · Cited by 46
“The final evaluation of a model behavior analysis is usually documented in the form of reports such as a checklist [51] or model card [20, 53].”
BackgroundMethodHow do Hugging Face Models Document Datasets, Bias, and Licenses? An Empirical Study
Pepe, F., Nardone, V., Mastropaolo, A., Bavota, G., Canfora, G., & Penta, M. D. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644412 · Cited by 38
Ramaswamy, V., Kim, S. S. Y., Fong, R., & Russakovsky, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01052 · Cited by 38
“…of works that propose checks and evaluation protocols to better understand the capabilities and limitations of interpretability methods [1–3, 21, 23, 26, 28 , 32, We also echo the call for releasing more information when releasing datasets [17], models [12, 33] and inter-pretability methods [47].”
BackgroundRidley, M. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24889 · Cited by 61
“” However, in an example where this proposal falls short of HCXAI, “ unintended uses, ethics, and safety are too ambiguous to be actionable ” and were not incorporated into the interactive data cards (Crisan et al., 2022, p. 437).”
Tan, M., & Subramonyam, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642592 · Cited by 40
“As studies have emphasized the importance of practitioner interaction with AI models and their data [7, 30], companies such as HuggingFace [1] and Google Cloud [2] have implemented interactive modalities for assessing model performance, though still targeting a technical audience.”
BackgroundHow Knowledge Workers Use and Want to Use LLMs in an Enterprise Context
Brachman, M., El-Ashry, A., Dugan, C., & Geyer, W. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650841 · Cited by 37
Stroud, A. M., Minteer, S. A., Zhu, X., Ridgeway, J. L., Miller, J. E., & Barry, B. A. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000826 · Cited by 43
“Visual and interactive aspects of model documentation could also be considered in support of patient needs, and further highlights the relevance of user interactions with AI systems when designing for transparency [12,54].”
Aspirations and Practice of ML Model Documentation: Moving the Needle with Nudging and Traceability
Bhat, A., Coursey, A., Hu, G., Li, S., Nahar, N., Zhou, S., Kästner, C., & Guo, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581518 · Cited by 33
“A more recent work built on the concept of model cards delves deep into how the content of model cards can support non-experts in making decisions related to the model [16].”
BackgroundMethodThe “Collections as ML Data” checklist for machine learning and cultural heritage
Lee, B. C. G. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24765 · Cited by 55
“3 Closely related work, such as “ Interactive Model Cards ” (Crisan et al., 2022), is worth noting as well.”
Applications of carbon-based diamond detectors: A critical review
Ahmad, U. F., Wudil, Y. S., Imam, A., Isa, N. F., Gondal, M. A., & Al-Osta, M. A. (2023) · Materials Today Communications · doi:10.1016/j.mtcomm.2023.106409 · Cited by 23
Improving Governance Outcomes Through AI Documentation: Bridging Theory and Practice
Winecoff, A., & Bogen, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713814 · Cited by 38
“Given the time constraints and the currently limited incentives for creating [17, 34, 48] and using [23] documentation, maintaining the core utility of documentation through standardization could support broader adoption within organizations.”
InfluentialFerenci, S., Coteț, F.-A., Lakatos, E. S., Munteanu, R. A., & Szabó, L. (2026) · Energies · doi:10.3390/en19020476 · Cited by 15
Manifestations of xenophobia in AI systems
Tomašev, N., Maynard, J. L., & Gabriel, I. (2024) · AI & Society · doi:10.1007/s00146-024-01893-4 · Cited by 22
“LLMs may potentially surface direct xenophobic discrimination, they may pose information hazard by revealing sensitive or private information about vulnerable individuals, spread misinformation that disparages out-groups, be incorporated in technologies weaponized by malicious xen-ophobic users,…”
A unified and practical user-centric framework for explainable artificial intelligence
Kaplan, S., Uusitalo, H. M. T., & Lensu, L. T. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111107 · Cited by 21
Co-Design Perspectives on Algorithm Transparency Reporting: Guidelines and Prototypes
Luria, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594064 · Cited by 16
“Some researchers have suggested improving documentation processes and mechanisms in the technical development process, such as using datasheets [28], model cards [19, 52] fact sheets [61].”
BackgroundEcosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14
Knowledge Workers' Perspectives on AI Training for Responsible AI Use
Zhang, A., & Lee, M. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714100 · Cited by 19
Current and Future Use of Large Language Models for Knowledge Work
Brachman, M., El-Ashry, A., Dugan, C., & Geyer, W. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757403 · Cited by 39
“Our study provides a foundation for future work designing tools to support workers using LLMs.”
Design, Development, and Deployment of Context-Adaptive AI Systems for Enhanced User Adoption
Lee, C. P. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3638195 · Cited by 15
“Efforts are directed towards enhancing transparency in technical models [28, 37], encompassing frameworks for documenting models and datasets [2, 4, 7], methods for elucidating model outputs [17, 31, 34], and strategies for effectively communicating uncertainty [5, 8].”
BackgroundA comprehensive review of techniques for documenting artificial intelligence
Königstorfer, F. (2024) · Digital Policy Regulation and Governance · doi:10.1108/dprg-01-2024-0008 · Cited by 15
InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations
Feldhus, N., Wang, Q., Anikina, T., Chopra, S., Oguz, C., & Möller, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.359 · Cited by 15
Turri, V., Morrison, K., Robinson, K.-M., Abidi, C., Perer, A., Forlizzi, J., & Dzombak, R. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658985 · Cited by 10
Ethical Framework for AI in Education
Farooqi, S. A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5550-4.ch007 · Cited by 11
Kaleidoscope: Semantically-grounded, context-specific ML model evaluation
Suresh, H., Shanmugam, D., Chen, T., Bryan, A. G., D’Amour, A., Guttag, J. V., & Satyanarayan, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581482 · Cited by 11
“In response, some work has proposed documenting performance across more granular data subgroups [8, 26, 35, 56].”
BackgroundTowards semantic versioning of open pre-trained language model releases on hugging face
Ajibode, A., Bangash, A. A., Cogo, F. R., Adams, B., & Hassan, A. E. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10631-3 · Cited by 19
“More recently, research has explored ML model and dataset documentation practices (Oreamuno et al., 2024, Mitchell et al., 2019, Wadhwani and Jain, 2020, Crisan et al., 2022, Casta˜no et al., 2024).”
Effectiveness and Information Quality Perception of an AI Model Card: A Study Among Non-Experts
Bracamonte, V., Pape, S., Löbner, S., & Tronnier, F. (2023) · Conference on Privacy, Security and Trust · doi:10.1109/pst58708.2023.10320197 · Cited by 10
“Crisan et al. [17] conducted semi-structured interviews with participants who had used machine learning or NLP model, in which they showed the participants a static model card and their proposed interactive version and asked them their opinion.”
BackgroundUsing Model Cards for ethical reflection on machine learning models: an interview-based study
Nunes, J. L., Barbosa, G. D. J., de Souza, C. S., & Barbosa, S. D. J. (2024) · Journal on Interactive Systems · doi:10.5753/jis.2024.3444 · Cited by 8
“Crisan et al. [2022] propose a direct expansion to the tool.”
BackgroundMaking the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection
Jung, J.-Y., Saxena, D., Park, M., Kim, J., Forlizzi, J., Holstein, K., & Zimmerman, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735745 · Cited by 9
Personalized LoRA for Human-Centered Text Understanding
Zhang, Y., Wang, J., Yu, L., Xu, D., & Zhang, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29931 · Cited by 16
“Human-centered text understanding (HCTU) aims to capture potential mental states in texts according to user preferences where user historical written texts are informative sources for user understanding (Crisan et al. 2022; Lynn et al. 2017; Capel and Brereton 2023).”
BackgroundAn Emerging Design Space of How Tools Support Collaborations in AI Design and Development
Muralikumar, M. D., & McDonald, D. W. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701181 · Cited by 7
“It enables lightweight experimentation and interrogation without the need to write custom code and thus, provides a more interpretable, accessible, and contextual way to understand the model [29].”
InfluentialRiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting
Rao, P. S. B., Šćepanović, S., Zhou, K., Bogucka, E. P., & Quercia, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713979 · Cited by 13
“Existing studies have investigated approaches to improve the usability and effectiveness of model cards such as introducing interactivity [15], adopting nudging formats like DocML [4], and treating AI documentation as a formal project deliverable [13] to encourage better documentation practices.”
InfluentialTowards a Non-Ideal Methodological Framework for Responsible ML
Mothilal, R. K., Guha, S., & Ahmed, S. I. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642501 · Cited by 14
“In addition, several prior works support, modify, and improve previously developed tools drawing on theories, empirical analysis, and the practical experiences of practitioners [23, 124].”
BackgroundRacial/Ethnic Categories in AI and Algorithmic Fairness: Why They Matter and What They Represent
Mickel, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659050 · Cited by 7
“Our work is motivated by previous scholarship on racial categories in algorithmic and AI fairness [1, 9, 25] as well as by existing documentation frameworks for datasets and models [8, 15, 17, 19, 24, 28, 30, 43, 56].”
MethodTrustworthiness Optimisation Process: A Methodology for Assessing and Enhancing Trust in AI Systems
Fikardos, M., Lepenioti, K., Apostolou, D., & Mentzas, G. N. (2025) · Electronics · doi:10.3390/electronics14071454 · Cited by 6
“Building on this foundation, Crisan et al. [38] proposed interactive model cards, enhancing user engagement through dynamic visualisations and detailed metadata.”
Model Cards Revisited: Bridging the Gap Between Theory and Practice for Ethical AI Requirements
Puhlfürß, T., Butzke, J., & Maalej, W. (2025) · IEEE International Requirements Engineering Conference · doi:10.1109/re63999.2025.00034 · Cited by 12
“Frameworks such as fact sheets [22] and explainability fact sheets [23] concentrate on system-level documentation, while model cards [9] and their interactive model card variant [24] focus on the models themselves.”
ALOHA: A(IBoM) tooL generatOr for Hugging fAce
D'Avino, R., Nocera, S., Bifolco, D., Pepe, F., Penta, M. D., & Scanniello, G. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756998 · Cited by 6
Large Language Models as SocioTechnical Systems
Dhole, K. (2023) · doi:10.18653/v1/2023.bigpicture-1.6 · Cited by 7
“For example, Crisan et al. (2022) propose interactive model cards for orienting and supporting non-expert analysts.”
BackgroundKotilainen, P., Mäkitalo, N., Systä, K., Mehraj, A., Waseem, M., Mikkonen, T., & Murillo, J. M. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112333 · Cited by 5
Patra ModelCards: AI/ML Accountability in the Edge-Cloud Continuum
Withana, S., & Plale, B. (2024) · IEEE International Conference on eScience · doi:10.1109/e-science62913.2024.10678710 · Cited by 5
“Crisan et al. [9] suggest Interactive Model Cards to make model documentation more transparent and accessible to a wider range of people.”
BackgroundShrestha, H., Cachel, K., Alkhathlan, M., Rundensteiner, E. A., & Harrison, L. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594108 · Cited by 4
“…highlight the inherent promise of interactive visualization for advancing goals in the fairness community, such as work from Mitchell et al. and Crisan et al. on model cards [15, 36], and Van Berkel et al. on examining the value of visualization over text for communicating fairness concepts [51].”
BackgroundIdentifying Algorithmic Decision Subjects' Needs for Meaningful Contestability
Yurrita, M., Verma, H., Balayn, A., Alfrink, K., Gadiraju, U., & Bozzon, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757415 · Cited by 11
“This could be operationalized by, for example, implementing explanations with hierarchies of information and varying levels of detail [28] or making explanations interactive [58].”
Toward Patient-Centered AI Fact Labels: Leveraging Extrinsic Trust Cues
Yoo, D. W., Stroud, A. M., Zhu, X., Miller, J. E., & Barry, B. A. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735758 · Cited by 7
“Notwithstanding, the model facts labels did not take into consideration patient perspectives.”
InfluentialCrisan, A., Butters, N., & Zoe (2024) · IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV) · doi:10.1109/beliv64461.2024.00007 · Cited by 3
“…[22, 70, 19], researchers prioritized identifying harms and risks [21, 17, 51, 75] as well as proposing frame-* works [49, 66], documentation [45, 13, 25], eXplainable Artificial Intelligence (XAI) techniques [54, 41, 64, 1], interactive visualization techniques [46, 76, 67, 62], and benchmarks…”
Self-citationGovernance of AI-Based Algorithms
Okolo, C. T. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_84 · Cited by 3
Khadka, R., Lind, P. G., Yazidi, A., & Belhadi, A. (2025) · SoftwareX · doi:10.1016/j.softx.2025.102140 · Cited by 6
“Crisan et al. [11] explored interactive model cards, which enhance traditional static model cards by allowing users to explore documentation and interact with models.”
Understanding Industry Practitioners' Experiences in Generative AI Governance
Do, H. J., Babbar, S., Li, W., Walks, L., & Misko, S. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720275 · Cited by 2
Porting an LLM based Application from ChatGPT to an On-Premise Environment
Paloniemi, T., Setälä, M., & Mikkonen, T. (2025) · International Conference on Software Reuse · doi:10.1109/icsr66718.2025.00014 · Cited by 4
“Transparency for model metadata has been advocated in studies like [22] and [23].”
A Conceptual Framework of AI Transparency From Sociotechnical Perspective
Huang, C., Wang, Z., Wen, X., Wang, X., & Yao, X. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3608735 · Cited by 2
“Crisan et al. [39] proposed an interactive model cards that augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves.”
Towards Explainable Automated Knowledge Engineering With Human-in-the-Loop
Zhang, B., Meroño-Peñuela, A., & Simperl, E. P. B. (2025) · Semantic Web · doi:10.1177/22104968251382171 · Cited by 4
“One potential solution is to document an interactive model card (Crisan et al., 2022) that lists all the necessary information regarding explainability.”
Trustworthy requirements for foundation models—A comprehensive survey and roadmap
Song, P., Ojo, A., & Curry, E. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.113111 · Cited by 5
Pepe, F., Nardone, V., Mastropaolo, A., Canfora, G., Bavota, G., & Penta, M. D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10843-1 · Cited by 2
Model ChangeLists: Characterizing Updates to ML Models
Eyuboglu, S., Goel, K., Desai, A., Chen, L., Monfort, M., Ré, C., & Zou, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659047 · Cited by 6
Designing Tiny Robots: Engaging Stakeholders in Collaborative AI System Design
Raftopoulos, M. (2024) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3681716.3681722 · Cited by 1
“The most effective means of working with these complex AI technology challenges is in association with stakeholders, and through novel collaborative and participatory approaches [13, 14].”
Talking About the Assumption in the Room
Mothilal, R. K., Lalani, F. M., Ahmed, S. I., Guha, S., & Sultana, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713958 · Cited by 2
“…practice continues to exist and is reflected in how the technical reports and documentation of large language models are currently drafted, such as in the responses to model cards [87] or datasheets [37] where there is less clarity on the reflectivity that goes into this process [11, 22, 79, 137].”
Cousineau, C., Herger, N., & Dara, R. A. (2025) · AI and Ethics · doi:10.1007/s43681-025-00725-5 · Cited by 1
Answering User Questions About Machine Learning Models Through Standardized Model Cards
Toma, T. R., Grewal, B., & Bezemer, C. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00066 · Cited by 3
Expanding ML-Documentation Standards For Better Security
Appel, C. E. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00041 · Cited by 1
“To aid developers, especially non-experts, in creating good documentation, interactive solutions analogous to those that are already common in non-ML documentation have been proposed which make standards more easily adoptable [10] [12].”
InfluentialTransparency in healthcare AI: Testing EU regulatory provisions against users’ transparency needs
Spagnolli, A., Tolomini, C., Beretta, E., Pluchino, P., & Sarra, C. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001594 · Cited by 4
“For example, Crisan et al. [50] suggest that notices and cards should include warnings, prompts, and summaries, which clarify how to act on the information provided.”
Anik, A. I., & Bunt, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789087 · Cited by 2
“As they build upon the dataset documentation literature [53], they share common ground with artifacts such as Data Cards [102] and Model cards [36, 90].”
Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Chappidi, S., Singh, J. P., & Krauze, A. V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791428 · Cited by 5
Triedman, H., Sarathy, J., Nanayakkara, P., Cummings, R., Kaptchuk, G., Kross, S., & Redmiles, E. M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791573 · Cited by 1
“Below, we describe our recruitment methods and participant sample, study protocol, and analysis methods.”
Responsible Model Selection with Virny and VirnyView
Herasymuk, D., Khan, F. A., & Stoyanovich, J. (2024) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3626246.3654738 · Cited by 1
Dzierzanowski, R., Watson, T., & Stevenson, C. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/bvyjt · Cited by 2
Position: Child Safety Necessitates New Approaches to AI SafetyPreprint
Kale, N., Portnoff, R. S., Thaker, P., Simpson, M., Wang, R., Kuo, K., Yadav, C., & Smith, V. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6206860 · Cited by 1
[WORKSHOP] Augmented Humans: Provocations for collaborative AI system design
Raftopoulos, M. (2023) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3616961.3616969
Evaluation of AI ethics tools in language models: a developers’ perspective case study
Silva, J., Moreira, D. A. B., dos Santos, G. O., Ferreira, A., Maia, H., Avila, S., & Pedrini, H. (2026) · AI and Ethics · doi:10.1007/s43681-025-00914-2 · Cited by 2
Evaluation Reporting in AI Model Releases should be StandardizedPreprint
Dhar, R., Villegas, D. S., Karamolegkou, A., Schiavone, A., Yuan, Y., Chen, X., Li, J., Frank, S., Grazia, L. D., Swain, M., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5444574
MRM3: Machine Readable ML Model Metadata
Čop, A., Bertalanič, B., Grobelnik, M., & Fortuna, C. (2025) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3711875.3736685
Experiment Cards: A Documentation Framework for Trustworthy AI Experiments
Gkioka, G., Apostolou, D., Verginadis, Y., & Mentzas, G. (2025) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai66417.2025.00065
“Over the past few years, initiatives like Model Cards[1], [2], [3], Datasheets for Datasets[4], [5], [6], [7], [8] and System Cards[9], [10] have provided standardized formats to document models, datasets, and deployed systems.”
Zhang, A., & Lee, M. K. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791306
PASTA: A Scalable Framework for Multi-Policy AI Compliance Evaluation
Yang, Y., Kim, I.-J., & Yoon, D. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790966
Gaballah, S. A., Cetinkaya, N. E., Wischnewski, M., & Sasse, M. A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791274 · Cited by 1
“Trying to increase multi-level transparency recommendations, approaches such as the AI Nutrition Facts [34], Open Ethic’s Nutrition Label [27], and energy labels [29], as well as data or model cards [23, 49, 52] have been introduced and tested.”
Wischnewski, M., Scharmann, A., Ridder, A., & Krämer, N. C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790983
“Although for professional contexts labeling approaches for AI are not new, for example, in the form of data or model cards [25, 76], AI labels for the general public have been less explored [36].”
Dai, D., Zhang, Y., Ma, Y., & Kragić, D. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791239
Légitime, S., & Suresh, H. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817010 · Cited by 1
An LLM-Based Approach for Automatic ML Prototype Review
Coban, S., Perez, M., Akpinar, C., Tanriverdi, B., & Lichter, H. (2026) · IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C) · doi:10.1109/saner-c67878.2026.00044
“Understandability [12], [13], [16], [22], [23], [26], [29]–[32] reflects whether the ML prototype can be comprehended by diverse stakeholders such as data scientists, software engineers, domain experts, or project managers.”
SPATA: Systematic Pattern Analysis for Detailed and Transparent Data Cards
Vitorino, J., Maia, E., Praça, I., & Soares, C. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19102-1_22
Who Defines “Best”? Towards Interactive, User-Defined Evaluation of LLM Leaderboards
Jung, M., Lee, M., Kim, Y., Choi, S., & Kahng, M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812353 · Cited by 2
Yi, H., Xian, L., Zhang, Y., Zeng, J., & Zhang, Z. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812239
Schmude, T., Pahr, D., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812271
When Efficiency Shapes Fairness: Cost-Optimised Human-in-the-Loop Routing
Breene, V., O'Hagan, A., & Costello, F. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812278
“Alongside interpretability, institutional risk management has evolved toward comprehensive diagnostic and monitoring frameworks that evaluate models across dimensions such as accuracy, stability, robustness, and fairness [14, 43].”
Beyond Collected Data: Surfacing and Controlling Shadow Inferences in Everyday Workflows
GUDIPATI, S. K., & MISHRA, N. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30047-8_2
Balayn, A., Rancourt, F., Casati, F., & Gadiraju, U. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816909
“Besides, most participants were not aware of research progress along these lines, such as simple or interactive AI documentation [39, 57, 105], interfaces [40, 153], or training programs [136, 138].”
Metadata
- Authors
- Crisan, A., Drouhard, M., Vig, J., & Rajani, N.
- Affiliations
- Tableau Software (United States) (US)
- Salesforce (United States) (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 427-439
- OpenAlex
- W4229447062
- Cited by
- 101
- References
- 56
- Citations by year
- 2026: 24 · 2025: 31 · 2024: 22 · 2023: 23 · 2022: 1
- FWCI
- 10.64
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Keywords
- model documentation, human-centered design, non-expert users, deep learning models, natural language processing, AI ethics, model interpretability, trust and safety, design guidelines, stakeholder perspectives, organizational dynamics, socio-technical context
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
