What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Computational notebooks - such as Azure, Databricks, and Jupyter - are a popular, interactive paradigm for data scientists to author code, analyze data, and interleave visualizations, all within a single document. Nevertheless, as data scientists incorporate more of their activities into notebooks, they encounter unexpected difficulties, or pain points, that impact their productivity and disrupt their workflow. Through a systematic, mixed-methods study using semi-structured interviews (n=20) and survey (n=156) with data scientists, we catalog nine pain points when working with notebooks. Our findings suggest that data scientists face numerous pain points throughout the entire workflow - from setting up notebooks to deploying to production - across many notebook environments. Our data scientists report essential notebook requirements, such as supporting data exploration and visualization. The results of our study inform and inspire the design of computational notebooks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bethlehem, J. (2010) · International Statistical Review · doi:10.1111/j.1751-5823.2010.00112.x · Cited by 1,315 · cited 1×
“In our survey, self-selection bias may have also influenced our results [3]: our respondents may feel more strongly about pain points in notebooks than the general population of data scientists.”
ResultBest Practices for Scientific Computing
Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001745 · Cited by 723
Automating string processing in spreadsheets using input-output examples
Gulwani, S. (2011) · ACM SIGPLAN Notices · doi:10.1145/1925844.1926423 · Cited by 581
Managing Messes in Computational Notebooks
Head, A., Hohman, F., Barik, T., Drucker, S. M., & DeLine, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300500 · Cited by 165 · cited 3×
“Researchers have developed some tools to improve the experience of computational notebooks, mostly around version con-trol and history [18, 23, 19], cleaning and annotating code [34, 13], working with streaming data [5], and extending the note-book model to make visualizations more central [39].”
BackgroundMethodSelf-citationExploring exploratory programming
Beth Kery, M., & Myers, B. A. (2017) · 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2017.8103446 · Cited by 123 · cited 1×
“Kery and colleagues [2] described exploratory programming—such as in computational notebooks—as a key practice for data scientists; they explored further the behaviors and characteristics of this work.”
Background
Cited by
How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study
Gu, K., Grunde-McLaughlin, M., McNutt, A., Heer, J., & Althoff, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641891 · Cited by 40
InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks
Lin, Y., Li, H., Yang, L., Wu, A., & Qu, H. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327170 · Cited by 40
“That is, how to facilitate results documentation which could interrupt the analysis flow and be discarded by users as being too demanding [9,36,46].”
BackgroundNotable: On-the-fly Assistant for Data Storytelling in Computational Notebooks
Li, H., Ying, L., Zhang, H., Wu, Y., Qu, H., & Wang, Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580965 · Cited by 38
Wang, A. Y., Wang, D., Drozdal, J., Liu, X., Park, S., Oney, S., & Brooks, C. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451617 · Cited by 31
“This known challenge of having incomplete or no documentation indeed hinders the readability and reusability of these notebooks, when they are shared with other collaborators or with one’s future self [2].”
BackgroundAyobi, A., Hughes, J., Duckworth, C. J., Dylag, J. J., Gordon James, S., Marshall, P., Guy, M., Kumaran, A., Chapman, A., Boniface, M., et al. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581424 · Cited by 26
“Drawing on semi-structured interviews and a survey with data scientists, Chattopadhyay et al. [16] identify several pain points, from importing different data sources and handling large code corpuses to managing confidentiality and security challenges.”
Metadata
- Authors
- Chattopadhyay, S., Prasad, I., Henley, A. Z., Sarma, A., & Barik, T.
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-12
- OpenAlex
- W3012094108
- Cited by
- 159
- References
- 5
- FWCI
- 54.02
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities. https://app.cochat.ai/papers/10.1145/3313831.3376729
- The paper
- Chattopadhyay, S., Prasad, I., Henley, A. Z., Sarma, A., & Barik, T. (2020). What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376729
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.