Cited by 165Software Engineering ResearchScientific Computing and Data ManagementData Visualization and Analytics

Managing Messes in Computational Notebooks

Open access
Head, Andrew, Hohman, Fred, Barik, Titus, Drucker, Steven M., DeLine, Robert·2019·Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Abstract

Data analysts use computational notebooks to write code for analyzing and visualizing data. Notebooks help analysts iteratively write analysis code by letting them interleave code with output, and selectively execute cells. However, as analysis progresses, analysts leave behind old code and outputs, and overwrite important code, producing cluttered and inconsistent notebooks. This paper introduces code gathering tools, extensions to computational notebooks that help analysts find, clean, recover, and compare versions of code in cluttered, inconsistent notebooks. The tools archive all versions of code outputs, allowing analysts to review these versions and recover the subsets of code that produced them. These subsets can serve as succinct summaries of analysis activity or starting points for new analyses. In a qualitative usability study, 12 professional analysts found the tools useful for cleaning notebooks and writing analysis code, and discovered new ways to use them, like generating personal documentation and lightweight versioning.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3290605.3300500

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. Steven P. Callahan, Juliana Freire, Emanuele Santos, Carlos E. Schei- degger, Cláudio T. Silva, and Huy T. Vo. 2006. VisTrails: Visualization meets Data Management. In Proceedings of the ACM International Con- ference on Management of Data. ACM, 745-747.
  2. Brian Hempel, Justin Lubin, Grace Lu, and Ravi Chugh. 2018. Deuce: A Lightweight User Interface for Structured Editing. In Proceedings of the ACM/IEEE International Conference on Software Engineering. ACM/IEEE, 654-664.
  3. Joshua Hibschman and Haoqi Zhang. 2015. Unravel: Rapid Web Appli- cation Reverse Engineering via Interaction recording, Source Tracing, and Library Detection. In Proceedings of the ACM Symposium on User Interface Software and Technology. ACM, 270-279.
  4. Joshua Hibschman and Haoqi Zhang. 2016. Telescope: Fine-Tuned Discovery of Interactive Web UI Feature Implementation. In Proceed- ings of the ACM Symposium on User Interface Software and Technology. ACM, 233-245.
  5. Mary Beth Kery, Amber Horvath, and Brad Myers. 2017. Variolite: Sup- porting Exploratory Programming by Data Scientists. In Proceedings of ACM Conference on Human Factors in Computing Systems. 1265-1276.
  6. Mary Beth Kery and Brad A. Myers. 2017. Exploring Exploratory Pro- gramming. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing. IEEE, 25-29.
  7. Adam Rule, Aurélien Tabard, and James D. Hollan. Data from: Exploration and Explanation in Computational Notebooks. https://doi.org/10.6075/J0JW8C39. · DOI

Cited by

Sorted by citations

Authors
Head, A., Hohman, F., Barik, T., Drucker, S. M., & DeLine, R.
Affiliations
  • University of California, Berkeley (US)
  • Georgia Institute of Technology (US)
  • Microsoft (United States) (US)
Venue
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2019-05-02
Pages
1-12
OpenAlex
W2941232686
Cited by
165
References
35
Citations by year
2026: 11 · 2025: 19 · 2024: 22 · 2023: 30 · 2022: 45 · 2021: 23 · 2020: 12 · 2019: 3
FWCI
16.40
Type
conference paper
Language
en
Open access
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
Keywords
computational notebooks, code versioning, code reuse, data analysis pipeline, code documentation, Jupyter notebooks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review