Cited by 23Software Engineering ResearchExpert finding and Q&A systemsSoftware Engineering Techniques and Practices

To Reuse or Not To Reuse?: A Framework and System for Evaluating Summarized Knowledge

Open access
Liu, Michael Xieyang, Kittur, Aniket, Myers, Brad A.·2021·Proceedings of the ACM on Human-Computer Interaction

Abstract

As the amount of information online continues to grow, a correspondingly important opportunity is for individuals to reuse knowledge which has been summarized by others rather than starting from scratch. However, appropriate reuse requires judging the relevance, trustworthiness, and thoroughness of others' knowledge in relation to an individual's goals and context. In this work, we explore augmenting judgements of the appropriateness of reusing knowledge in the domain of programming, specifically of reusing artifacts that result from other developers' searching and decision making. Through an analysis of prior research on sensemaking and trust, along with new interviews with developers, we synthesized a framework for reuse judgements. The interviews also validated that developers express a desire for help with judging whether to reuse an existing decision. From this framework, we developed a set of techniques for capturing the initial decision maker's behavior and visualizing signals calculated based on the behavior, to facilitate subsequent consumers' reuse decisions, instantiated in a prototype system called Strata. Results of a user study suggest that the system significantly improves the accuracy, depth, and speed of reusing decisions. These results have implications for systems involving user-generated content in which other users need to evaluate the relevance and trustworthiness of that content.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3449240

Cites

113 of 114 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • This aligns with prior work emphasizing the importance of concise memory structures to avoid overloading the system and context window limitations.

    “This aligns with prior work emphasizing the importance of concise memory structures to avoid overloading the system and context window limitations [35, 36, 61].”

    From CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents (2026) · cites this paper as [36]

    Section 5.1 (p. 16) reports that showing all information at once overwhelmed users, and Section 9 (p. 28) describes a hierarchical overview that lets them explore details selectively. This supports the general idea of structuring information to reduce overload, but the paper addresses human attention in a user interface, not concise memory structures, system overload, or context-window limits. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [15][15] Janet E. Alexander and Marsha A. Tate. 1999. Web Wisdom; How to Evaluate and Create Information Quality on the Webb (1st ed.). L. Erlbaum Associates Inc., USA. -
  2. [30][30] Joseph Chee Chang, Nathan Hahn, Adam Perer, and Aniket Kittur. 2019. SearchLens: composing and capturing complex user interests for exploratory search. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI '19) . Association for Computing Machinery, Marina del Ray, California, 498-509. https://doi.org/10. 1145/3301275.3302321 -
  3. [53][53] Udo Hahn and Ulrich Reimer. 1999. Knowledge-based text summarization: Salience and generalization operators for knowledge base abstraction. Advances in automatic text summarization (1999), 215-232. Publisher: MIT Press, Cambridge, Mass. -
  4. [55][55] Johan F. Hoorn and Teunis D. van Wijngaarden. 2010. Web Intelligence for the Assessment of Information Quality: Credibility, Correctness, and Readability. Web Intelligence and Intelligent Agents (March 2010). https://doi.org/10. 5772/8372 Publisher: IntechOpen. -
  5. [68][68] Professor of Management and Director at the Institute of Management Georg Von Krogh, Georg von Krogh, Associate Professor in the Faculty of Social Sciences and the Graduate School of International Corporate Strategy Kazuo Ichijo, Kazuo Ichijo, Ikujiro Nonaka, and Professor of Graduate School of International Corporate Strategy at Hitotsubashi University and the Xerox Distinguished Professor in Knowledge at Hass School of Business Ikujiro Nonaka. 2000. Enabling Knowledge Creation: How to Unlock the Mystery of Tacit Knowledge and Release the Power of Innovation . Oxford University Press, USA. Google-Books-ID: JVESDAAAQBAJ. -
  6. [81][81] Michael Xieyang Liu, Shaun Burley, Emily Deng, Angelina Zhou, Aniket Kittur, and Brad A. Myers. 2018. Supporting Knowledge Acceleration for Programming from a Sensemaking Perspective. Sensemaking Workshop at CHI Conference on Human Factors in Computing Systems (April 2018). https://par.nsf.gov/biblio/10152063-supporting-knowledgeacceleration-programming-from-sensemaking-perspective -
  7. [82][82] Michael Xieyang Liu, Nathan Hahn, Angelina Zhou, Shaun Burley, Emily Deng, Aniket Kittur, and Brad A. Myers. 2018. UNAKITE: Support Developers for Capturing and Persisting Design Rationales When Solving Problems Using Web Resources. Workshop on Designing Technologies to Support Human Problem Solving at the IEEE Symposium on Visual Languages and Human-Centric Computing (Oct. 2018). https://par.nsf.gov/biblio/10152060-unakite-supportdevelopers-capturing-persisting-design-rationales-when-solving-problems-using-web-resources -

Cited by

Sorted by citations

Authors
Liu, M. X., Kittur, A., & Myers, B. A.
Affiliations
Carnegie Mellon University (US)
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2021-04-13
Volume
5
Issue
CSCW1
Pages
1-35
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3158751339
Cited by
23
References
114
Citations by year
2026: 4 · 2025: 5 · 2024: 5 · 2023: 3 · 2022: 6
FWCI
3.17
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
NSF (CCF-1814826)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
knowledge reuse, sensemaking, trustworthiness, decision visualization, user-generated content, strata, developer interviews

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review