The DOI matches its Crossref record, with a small difference in title or year; 89 of the 89 references it cites were found in a registry or an index; 21 of the 46 sources its claims cite were checked for support.
Progressive disclosure
Open accessAbstract
As we increasingly delegate important decisions to intelligent systems, it is essential that users understand how algorithmic decisions are made. Prior work has often taken a technocentric approach to transparency. In contrast, we explore empirical user-centric methods to better understand user reactions to transparent systems. We assess user reactions to transparency in two studies. In Study 1, users anticipated that a more transparent system would perform better, but retracted this evaluation after experience with the system. Qualitative data suggest this arose because transparency is distracting and undermines simple heuristics users form about system operation. Study 2 explored these effects in depth, suggesting that users may benefit from initially simplified feedback that hides potential system errors and assists users in building working heuristics about system operation. We use these findings to motivate new progressive disclosure principles for transparency in intelligent systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
28 Own result · 24 Own interpretation · 8 Partly supported · 8 Own method · 6 Supported · 5 No full text of the sources · 5 No basis given · 3 Not checked yet · 2 Couldn’t confirm · 1 Supported so far · accuracy not yet measured
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Further, making accurate predictions about emotions is also difficult, allowing us to examine a key challenge for intelligent systems: how to deal with system error.”
- Also stated in: § 2.4 Emotional Analytics · boosted
“previous research demonstrates difficulty in accurately predicting emotion from text [47,69].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites [47] 47. Shoushan Li, Lei Huang, Rong Wang, and Guodong Zhou. 2015. Sentence-level Emotion Classification with Label and Context Dependence. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 1045-1053. Retrieved September 21, 2018 from http://www.aclweb.org/anthology/P15-1101 source
- Supportedcites [69] 69. Lena Reed, Jiaqi Wu, Shereen Oraby, Pranav Anand, and Marilyn Walker. 2017. Learning Lexico-Functional Patterns for First-Person Affect. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 141-147. Retrieved September 18, 2018 from http://aclweb.org/anthology/P17-2022 source
The Introduction identifies a central obstacle to reliable affect prediction: people often describe situations rather than explicitly state their feelings. The Abstract likewise says existing sentiment dictionaries lack information needed to infer those reactions. This supports the citing sentence’s broad point that accurate emotion prediction is difficult, particularly for first-person accounts. (LLM, conf=0.90)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 2.4 Emotional Analytics · boosted
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“It can also promote user learning and insight from complex data, which is important as humans increasingly work with complex inferential systems for analytic purposes [43,71].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 1 Introduction
What the sources say
- Partly supportedcites [43] 43. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces -IUI '15, 126-137. https://doi.org/10.1145/2678025.2701399 sourceExperimental — accuracy not yet measured
The Related Work section (p. 3) says transparency helps users understand how a learning system operates, and the Conclusion (p. 11) reports improved understanding among users of Explanatory Debugging. The paper does not establish that transparency gives users insight into complex data for analytic purposes; its measured learning concerns the system itself. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [71] 71. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-1144. https://doi.org/10.1145/2939672.2939778 source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Some research suggests that the way we present transparency may account for these contradictory results [26,42].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 1 Introduction
What the sources say
- Partly supportedcites [26] 26. Pedro García Garcia, Enrico Costanza, Jhim Verame, Diana Nowacka, and Sarvapali D. Ramchurn. 2018. Seeing (Movement) is Believing: The Effect of Motion on Perception of Automatic Systems Performance. Human- Computer Interaction 0, 0: 1-51. https://doi.org/10.1080/07370024.2018.1453815 sourceExperimental — accuracy not yet measured
Section 2.2 (page unavailable) says explanations can help users understand smart systems but can also overwhelm or confuse them; it explores animation as another way to present system activity. This supports the idea that presentation matters, but the paper does not say that presentation accounts for contradictory findings about transparency’s effects; it also identifies system uncertainty as a reason effects differ. (LLM, conf=0.78)
- Not checked — no full text of the sourcecites [42] 42. René F. Kizilcec. 2016. How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface. 2390- 2395. https://doi.org/10.1145/2858036.2858402 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Consistent with our initial expectations and the prior literature on transparency [26,48], users anticipated a preference for transparency before using the system.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.3 Discussion
What the sources say
- Could not verifycites [26] 26. Pedro García Garcia, Enrico Costanza, Jhim Verame, Diana Nowacka, and Sarvapali D. Ramchurn. 2018. Seeing (Movement) is Believing: The Effect of Motion on Perception of Automatic Systems Performance. Human- Computer Interaction 0, 0: 1-51. https://doi.org/10.1080/07370024.2018.1453815 sourceExperimental — accuracy not yet measured
Section 2.2 discusses how explanations can help users understand smart systems, but it does not report that users anticipated preferring transparency before use. Section 3.4 describes a questionnaire before the task and evaluations after participants saw the systems; it does not report a pre-use transparency preference. Searches for that preference and its timing found no supporting passage in our copy. (LLM, conf=0.72)
- Not checked — no full text of the sourcecites [48] 48. Brian Y. Lim and Anind K. Dey. 2009. Assessing demand for intelligibility in context-aware applications. In Proceedings of the 11th international conference on Ubiquitous computing, 195-204. Retrieved May 8, 2017 from http://dl.acm.org/citation.cfm?id=1620576 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Transparency can also enable oversight by system designers.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 1 Introduction
What the sources say
- Partly supportedcites [33] 33. Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of Opportunity in Supervised Learning. arXiv:1610.02413 [cs]. Retrieved from http://arxiv.org/abs/1610.02413 sourceExperimental — accuracy not yet measured
The Summary of Contributions (p. 2) describes an interpretable, easily checkable framework, and the Conclusions (p. 20) say it can reveal concerns requiring further scrutiny. That supports the possibility of oversight, but the paper does not specifically say that transparency enables oversight by system designers. (LLM, conf=0.75)
- Supportedcites [51] 51. Zachary C. Lipton. 2016. The Mythos of Model Interpretability. arXiv:1606.03490 [cs, stat]. Retrieved September 21, 2018 from http://arxiv.org/abs/1606.03490 source
Section 3.1 defines transparency as understanding how a model works, and Section 3.1.2 describes how its components can admit explanations. Section 5 says practitioners and researchers share responsibility for accounting for machine learning’s impact. Together, these passages support the modest claim that transparency can enable oversight by those designing systems; the indexed copy provides no page numbers. (LLM, conf=0.82)
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These methods offer ways to collect controlled user data at scale [26,49].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.2 Impact for Future Transparency Research
What the sources say
- Partly supportedcites [26] 26. Pedro García Garcia, Enrico Costanza, Jhim Verame, Diana Nowacka, and Sarvapali D. Ramchurn. 2018. Seeing (Movement) is Believing: The Effect of Motion on Perception of Automatic Systems Performance. Human- Computer Interaction 0, 0: 1-51. https://doi.org/10.1080/07370024.2018.1453815 sourceExperimental — accuracy not yet measured
The Methodology and Procedure sections describe simulated system tasks with standardized conditions, including studies conducted through MTurk, so the paper supports collecting user data through such methods. But Participants says recruitment was deliberately limited to eight people per condition, and Study 2 calls crowdsourcing less controlled than a lab; the paper does not establish collection of controlled data at scale. Page numbers are unavailable in our copy. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [49] 49. Brian Y. Lim and Anind K. Dey. 2011. Investigating intelligibility for uncertain context-aware applications. In Proceedings of the 13th international conference on Ubiquitous computing, 415-424. Retrieved April 25, 2017 from http://dl.acm.org/citation.cfm?id=2030168 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our qualitative analysis of experiencebased preference suggests that users who know less about the working of the system, seem to ascribe more advanced abilities to it.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“If the algorithm was behaving consistently with their overall expectations, users in the non-transparent condition judged it very positively.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Given only retrospective accuracy scores from both versions, we can predict the version choice with 69.5% accuracy in a 5-fold cross-validated test.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This did not seem to be the case in the transparent condition, users judged the system more negatively even if the overall prediction was correct.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 22
- Probably not a substantial claim. 23
- The quote only partly states this claim. 1
Every source the paper cites for this claim was checked and supports it.
In the paper
“In prior experiments, users judged the same algorithm to be accurate but not perfect; the median user response for accuracy was 'Accurate' (6 on a 7-point Likert scale with the highest rating ('7') being 'Very Accurate') [78].”
- Also stated in: § 3.1 Machine Learning Model · plain
“As we noted above, previous users found the system to be accurate to their experience, judging median accuracy to be 6 out of 7 on a scale where 7 is 'Very Accurate'.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 3.1 Machine Learning Model
What the sources say
- Supportedcites [78] 78. Aaron Springer and Steve Whittaker. 2018. What are You Hiding? Algorithmic Transparency and User Perceptions. In 2018 AAAI Spring Symposium Series. source
The Results (p. 3) state that the median user rating of the E-meter was 'Accurate,' rather than 'Very Accurate.' The Survey (p. 2) describes a seven-point accuracy scale ending at 'Very Accurate,' consistent with the claim's scale description. (LLM, conf=0.89)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 3.1 Machine Learning Model · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While such a model may be less accurate than state of the art methods using deep neural networks [19], it provides a straightforward operationalization of transparency that is intuitively understandable while still appearing accurate and convincing to users [78].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 3.1 Machine Learning Model
What the sources say
- Partly supportedcites [19] 19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs]. Retrieved December 27, 2018 from http://arxiv.org/abs/1810.04805 sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
- Partly supportedcites [78] 78. Aaron Springer and Steve Whittaker. 2018. What are You Hiding? Algorithmic Transparency and User Perceptions. In 2018 AAAI Spring Symposium Series. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The E-meter system and protocol have been successfully deployed in multiple previous experiments [76,78].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 3 Research System: E-meter
What the sources say
- Partly supportedcites [76] 76. Aaron Springer, Victoria Hollis, and Steve Whittaker. 2017. Dice in the Black Box: User Experiences with an Inscrutable Algorithm. Retrieved April 24, 2017 from https://aaai.org/ocs/index.php/SSS/SSS17/paper/view/1537 2 sourceExperimental — accuracy not yet measured
The E-meter System and Materials and Methodology sections (p. 2) describe the system and procedure used with participants, supporting this paper as an example of a prior deployment. The Discussion (p. 3) calls it a small pilot; this source does not establish that the E-meter system and protocol were deployed in multiple previous experiments. (LLM, conf=0.84)
- Partly supportedcites [78] 78. Aaron Springer and Steve Whittaker. 2018. What are You Hiding? Algorithmic Transparency and User Perceptions. In 2018 AAAI Spring Symposium Series. sourceExperimental — accuracy not yet measured
The Users section (p. 2) reports that 41 users tested the E-meter system, and the E-meter System section (p. 2) describes the protocol. This supports its deployment in a previous experiment, but the paper describes two conditions within that study, not multiple previous experiments using the system and protocol. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Progressive disclosure has a long history in UI research, dating back to the Xerox Star and early word processing systems [14,61,74].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
What the sources say
- Partly supportedcites [14] 14. John M. Carroll and Caroline Carrithers. 1984. Training Wheels in a User Interface. Commun. ACM 27, 8: 800-806. https://doi.org/10.1145/358198.358218 sourceExperimental — accuracy not yet measured
The Training Wheels Interface and Discussion sections describe a 1984 word-processor interface that made basic functions available while withholding advanced ones, and call this approach “staging” the presentation of functions. This supports the early word-processing part of the claim, but our copy does not connect the approach to the Xerox Star. (LLM, conf=0.86)
- Partly supportedcites [61] 61. Lloyd H. Nakatani and John A. Rohrlich. 1983. Soft machines: A philosophy of user-computer interface design. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems -CHI '83, 19-23. https://doi.org/10.1145/800045.801573 sourceExperimental — accuracy not yet measured
The “Operability with Flexibility” section explicitly identifies progressive disclosure as characteristic of the Xerox Star interface. The paper mentions dedicated word processors elsewhere, but does not link early word-processing systems to progressive disclosure (page numbers unavailable). (LLM, conf=0.86)
- Not matchedcites [74] 74. David Canfield Smith. Designing the Star User Interface. 21.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Greater transparency potentially increases end user control and improves acceptance of complex algorithmic systems [43].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 1 Introduction
What the sources say
- Supportedcites [43] 43. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces -IUI '15, 126-137. https://doi.org/10.1145/2678025.2701399 source
Related Work (p. 3) associates transparency with greater user acceptance and says understanding a learning system helps users personalize it. The Conclusion (p. 11) describes the paper’s explanation-based approach as more controllable and satisfying than black-box labeling. These passages support the hedged claim in the context of machine-learning systems. (LLM, conf=0.90)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The original concept involved hiding advanced interface controls; allowing users to make fewer initial errors and learn the system more effectively [14].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
What the sources say
- Partly supportedcites [14] 14. John M. Carroll and Caroline Carrithers. 1984. Training Wheels in a User Interface. Commun. ACM 27, 8: 800-806. https://doi.org/10.1145/358198.358218 sourceExperimental — accuracy not yet measured
The Training Wheels Interface section says advanced functions were blocked while basic functions remained available; the abstract reports faster learning and better achievement. But advanced menu choices were not hidden: users could select them and receive an unavailable-function message. The paper supports preventing troublesome error states, not necessarily users making fewer initial mistakes. (LLM, conf=0.88)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“This replicates our previous work [78] where the median user found the E-meter to be 'Accurate'.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
What the sources say
- Supportedcites [78] 78. Aaron Springer and Steve Whittaker. 2018. What are You Hiding? Algorithmic Transparency and User Perceptions. In 2018 AAAI Spring Symposium Series. source
The Results (p. 3) state directly that, across conditions, the median user rated the E-meter's accuracy as 'Accurate'. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“For example, other researchers have examined transparency in the context of deep learning models predicting patient outcomes in the medical realm [45].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
What the sources say
- Supportedcites [45] 45. B. C. Kwon, M. Choi, J. T. Kim, E. Choi, Y. B. Kim, S. Kwon, J. Sun, and J. Choo. 2018. RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records. IEEE Transactions on Visualization and Computer Graphics: 1-1. https://doi.org/10.1109/TVCG.2018.2865027 source
The Introduction (p. 1) says the study addresses interpretability of an RNN-based model that predicts patients’ future risk of heart failure and cataract from electronic medical records. This supports the citing sentence’s description of research on transparency in deep-learning models predicting medical outcomes. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Each feature can be visualized and ranked for its contribution to the overall predicted diagnosis risk score.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
What the sources say
- Supportedcites [45] 45. B. C. Kwon, M. Choi, J. T. Kim, E. Choi, Y. B. Kim, S. Kwon, J. Sun, and J. Choo. 2018. RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records. IEEE Transactions on Visualization and Computer Graphics: 1-1. https://doi.org/10.1109/TVCG.2018.2865027 source
Patient Details (p. 6) says the temporal code chart visualizes contribution scores for all medical codes alongside predicted diagnosis risks, and its bar chart shows the top nine contributing codes. The paper treats medical codes as input features whose contribution scores indicate their impact on predictions. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“One method used in prior studies is to provide potential system users with hypothetical scenarios describing system operation and eliciting reactions to those systems.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.2 Impact for Future Transparency Research
What the sources say
- Could not verifycites [26] 26. Pedro García Garcia, Enrico Costanza, Jhim Verame, Diana Nowacka, and Sarvapali D. Ramchurn. 2018. Seeing (Movement) is Believing: The Effect of Motion on Perception of Automatic Systems Performance. Human- Computer Interaction 0, 0: 1-51. https://doi.org/10.1080/07370024.2018.1453815 sourceExperimental — accuracy not yet measured
Section 3.4 (page unavailable) describes participants viewing system interfaces and evaluating their performance, rather than reacting to hypothetical scenarios. Section 2.2 discusses prior studies of explanations and system intelligibility, but does not describe the claimed scenario-based method. Searches for hypothetical scenarios and vignettes found no passage establishing it. (LLM, conf=0.72)
- Not checked — no full text of the sourcecites [49] 49. Brian Y. Lim and Anind K. Dey. 2011. Investigating intelligibility for uncertain context-aware applications. In Proceedings of the 13th international conference on Ubiquitous computing, 415-424. Retrieved April 25, 2017 from http://dl.acm.org/citation.cfm?id=2030168 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“In addition, users can compare outcomes between patients and conduct what-if analyses by modifying patient attributes and seeing how predicted risk changes.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
What the sources say
- Supportedcites [45] 45. B. C. Kwon, M. Choi, J. T. Kim, E. Choi, Y. B. Kim, S. Kwon, J. Sun, and J. Choo. 2018. RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records. IEEE Transactions on Visualization and Computer Graphics: 1-1. https://doi.org/10.1109/TVCG.2018.2865027 source
The Patient List section (p. 6) says users can compare multiple patients and view each patient's predicted diagnosis risk. The Patient Editor section (p. 6) says users can change visits and medical codes to test hypothetical scenarios, then see the updated predicted risk. (LLM, conf=0.91)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Study 1, users anticipated that a more transparent system would perform better, but retracted this evaluation after experience with the system.”
- Also stated in: § 1.1 Contribution · plain
“In Study 1, users anticipated that a transparent system would perform better, but retracted this evaluation after experience with the system.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 1.1 Contribution · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using a train/test split of 85/15 the linear regression tested at R 2 = 0.25; mean absolute error was .95 on the target variable (mood) scale of (-3,3).”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 3.1 Machine Learning Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, the median user found the E-meter to be 'Slightly Accurate' and 'Slightly Trustworthy'.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we had originally anticipated, some users continued to prefer the transparent version and they were likely to cite the increased insight that it facilitated into the algorithm's underlying operation.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.3 Discussion
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some of these users seemed to find highlighting to be distracting, although our cognitive load results do not support this.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.3 Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both retrospective accuracy and subjective trust distributions were bimodal (due to the 7-point Likert item containing an infrequently used neutral item)”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“there was no difference between conditions (p = .24,p=.41).”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Qualitative user comments supported these anticipated benefits.”
- Also stated in: § System Perceived as Moderately Accurate and Trustworthy: · plain
“users drew attention to the benefits of transparent color-coded feedback giving a clearer sense of how the system was operating and boosting their confidence that the system was operating appropriately.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § System Perceived as Moderately Accurate and Trustworthy: · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, some users (like those in Study 1) may want yet more transparency.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, results from Study 1 indicate the importance of direct user experience when making system evaluations; users' perceptions of the system were very different following actual usage compared with their projected reactions prior to usage.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.2 Impact for Future Transparency Research
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Following prior methodological recommendations in [18], we eliminated 26 respondents based on their responses to open ended questions, leaving us with a sample of 74 users.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.1.1 Users.
What the sources say
- Not checked — no full text of the sourcecites [18] 18. Michael A. DeVito, Jeremy Birnholtz, Jeffery T. Hancock, Megan French, and Sunny Liu. 2018. How People Form Folk Theories of Social Media Feeds and What it Means for How We Study Self-Presentation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 1-12. https://doi.org/10.1145/3173574.3173694 source
plainn = 26- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The study design was counterbalanced; half the users experienced the non-transparent system version first.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.1 Method
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This study was approved by an Institutional Review Board.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.1.1 Users.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Questions and procedure were carefully piloted and had been used before in multiple prior studies.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.1 Method
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Qualitative data suggest this arose because transparency is distracting and undermines simple heuristics users form about system operation.”
- Also stated in: § 1.1 Contribution · hedged
“Qualitative data suggest this may be because incremental transparency feedback is distracting, potentially undermining simple heuristics users form of system operation.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > Abstract
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 1.1 Contribution · hedged
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We used Latin text because we wanted high-level system comparisons that were not based on users' reactions to specific English words.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.1.2 Measures.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even if the system is objectively correct when displaying transparency information, it can still cause users to take issue and result in poorer perceptions of that system.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > Transparency May Violate User Expectations Even
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Following recommendations from an IRB panel, the PGWBI screening was intended to exclude users who might find the emotional reflection distressing.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 4.1.1 Users.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The absence of real user feedback makes it challenging to operationalize transparency in ways that positively impact users.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 2.2 Transparency
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“As with other areas of reasoning, it may be that in interacting with a system users first engage in rapid, approximate, System 1 thinking and only engage in deeper, more analytic, System 2 thinking when directly prompted or confronted with clearly anomalous information [40,66].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > User Heuristics May Contradict Transparency:
What the sources say
- Not matchedcites [40] 40. Daniel Kahneman. 2011. Thinking, Fast and Slow. Macmillan.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [66] 66. Richard E. Petty and John T. Cacioppo. 1986. The Elaboration Likelihood Model of Persuasion. Advances in IUI'19, March, 2019, Marina Del Rey, CA USA Aaron Springer and Steve Whittaker Experimental Social Psychology 19: 123-205. https://doi.org/10.1016/S0065-2601(08)60214-2 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While the machine learning model was limited to the 600 most predictive words of mood, that model still seemed to present too many extraneous words that users felt were unimportant.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > More Transparency is Not Always Better:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One possibility is that non-transparent feedback can hide low-level errors from users.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Recall that both systems use the same underlying machine learning model which uses solely individual word features.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, our deployment of a working algorithm meant that results were obtained for situations where our algorithm generated moderate numbers of errors-future research should compare contexts where there are different levels of errors [46,47].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.3 Limitations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note too that because transparency is provided 'on demand' this removes confusions and inefficiencies arising from spurious, unwanted explanations, and adjusts explanations to the users' requirements.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, while users generated their own data in our system, results were not directly used to inform other aspects of the user's personal behavior so the costs of system errors were low.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.3 Limitations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There are many other ways to depict how an algorithm operates including verbal explanations, concrete user exploration, and so forth [42,44,50,71].”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.3 Limitations
What the sources say
- Not checked — no full text of the sourcecites [42] 42. René F. Kizilcec. 2016. How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface. 2390- 2395. https://doi.org/10.1145/2858036.2858402 source
- Not checked — no full text of the sourcecites [44] 44. Todd Kulesza, Weng-Keen Wong, Simone Stumpf, Stephen Perona, Rachel White, Margaret M. Burnett, Ian Oberst, and Andrew J. Ko. 2008. Fixing the program my computer learned: barriers for end users, challenges for the machine. In Proceedingsc of the 13th international conference on Intelligent user interfaces -IUI '09, 187. https://doi.org/10.1145/1502650.1502678 source
- Not checked — no full text of the sourcecites [50] 50. Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the 27th international conference on Human factors in computing systems - CHI 09, 2119. https://doi.org/10.1145/1518701.1519023 source
- Not checked — no full text of the sourcecites [71] 71. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-1144. https://doi.org/10.1145/2939672.2939778 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The current study indicates a promising design approach involving progressive disclosure which we intend to explore in future work.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.4 Conclusion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Progressive disclosure of transparency is not limited to predictions from text.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Again, the major contribution of progressive disclosure is avoiding overwhelming users with information, but instead slowly increasing transparency as users indicate a willingness to engage meaningfully with it.”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > 6.1 Meeting the Competing Needs of Transparency Through Progressive Disclosure
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
Users are the ultimate experts on their own emotions, providing them with ground truth assessments about whether the system is correct.
“And users are the ultimate experts on their own emotions, providing them with ground truth assessments about whether the system is correct.”
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(73)=.910,p=.343”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
Reported p = .343 · recomputed p = .366
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(73) = -.05,p=.95”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
Reported p = .95 · recomputed p = .960
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(73)=5.452,p=0.022”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
Reported p = .022 · recomputed p < .001
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upt(73) = 5.452reported p = .022 · recomputed p < .001
“t(73)=5.452,p = .022”
- Doesn’t add upt(73) = 0.91reported p = .343 · recomputed p = .366
“t(73)=.910,p = .343”
- Doesn’t add upt(73) = -0.05reported p = .95 · recomputed p = .960
“t(73) = -.05,p = .95”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
“t(73)=5.452”
§ Progressive Disclosure Empirically Motivated Approaches to Designing Effective Transparency > System Perceived as Moderately Accurate and Trustworthy:
“p=0.022”
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587 · cited 1×
“One possibility is that these differences arise from individual user traits, such as need for control [2] or need for cognition [13].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 2×
“These approaches often rely on approximating the inscrutable algorithm through a simple local or linear model than can be explained to the end user [53,71].”
MethodGrice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221 · cited 2×
“One approach is to model causal explanation as a form of conversation which is governed by common-sense conversational rules [36] such as Grice’s maxims [30].”
BackgroundResult"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 5×
“These approaches often rely on approximating the inscrutable algorithm through a simple local or linear model than can be explained to the end user [53,71].”
BackgroundInfluentialDevelopment of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 1×
“After this system experience, users completed the TLX workload assessment [34].”
Method(2011) · Cited by 14,317
Wallace, A. F. C., & Garfinkel, H. (1968) · American Sociological Review · doi:10.2307/2092245 · Cited by 12,413
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 1×
“For example, seemingly random noise that is unnoticeable to humans can cause deep neural network image classifiers to fail catastrophically [28].”
Background
Research works printed but not linked: 10
- Angwin Julia, 2016, ProPublica. Retrieved
- Devlin Jacob, 2018, Retrieved
- Lakkaraju Himabindu, 2017, Retrieved
- Lipton Zachary C., 2016, Retrieved
- Miller Tim, 2017, Retrieved
- Springer Aaron, 2017, Retrieved
- Springer Aaron, 2018, Algorithmic Transparency and User Perceptions. In 2018 AAAI Spring Symposium Series.
- Daniel, 2018, Retrieved
- Zisook Miriam, 2016, Retrieved
- Woebot, 2018, Retrieved
Cited by
Sorted by citations
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320
“and progressively [54, 62, 69], and methods for incorporating the consideration of cognitive load into the explanation generation process [1] should be explored.”
MethodTowards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
“Understanding subjective [7, 12, 13, 14, 16, 17, 22, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44] objective [12, 13, 22, 32, 35, 39, 40, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60] explanation model [21, 46, 49, 61, 62, 63, 64, 65]”
BackgroundInfluentialEvaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163
“Springer and Whiaker [55] conducted two user studies.”
BackgroundThe Placebo Effect of Artificial Intelligence in Human–Computer Interaction
Kosch, T., Welsch, R., Chuang, L., & Schmidt, A. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3529225 · Cited by 137
Metadata
- Authors
- Springer, A., & Whittaker, S.
- Affiliations
- University of California, Santa Cruz (US)
- Venue
- Proceedings of the 24th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2019-03-17
- Pages
- 107-120
- OpenAlex
- W2918066172
- Cited by
- 115
- References
- 87
- Citations by year
- 2026: 19 · 2025: 17 · 2024: 11 · 2023: 17 · 2022: 15 · 2021: 14 · 2020: 16 · 2019: 2
- FWCI
- 4.50
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- progressive disclosure, transparency, algorithmic decision-making, intelligent systems, system errors, user distraction, user-centered design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
