MixT: automatic generation of step-by-step mixed media tutorials
The DOI matches its Crossref record; 21 of the 22 references it cites were found in a registry or an index.
Abstract
Users of complex software applications often learn concepts and skills through step-by-step tutorials. Today, these tutorials are published in two dominant forms: static tutorials composed of images and text that are easy to scan, but cannot effectively describe dynamic interactions; and video tutorials that show all manipulations in detail, but are hard to navigate. We hypothesize that a mixed tutorial with static instructions and per-step videos can combine the benefits of both formats. We describe a comparative study of static, video, and mixed image manipulation tutorials with 12 participants and distill design guidelines for mixed tutorials. We present MixT, a system that automatically generates step-by-step mixed media tutorials from user demonstrations. MixT segments screencapture video into steps using logs of application commands and input events, applies video compositing techniques to focus on salient infor-mation, and highlights interactions through mouse trails. Informal evaluation suggests that automatically generated mixed media tutorials were as effective in helping users complete tasks as tutorials that were created manually.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Furnas, G. W. (1986) · ACM SIGCHI Bulletin · doi:10.1145/22339.22342 · Cited by 1,629
Furnas, G. W. (1986) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/22627.22342 · Cited by 890 · cited 1×
“This video composition effectively creates a focus-plus-context view that makes important regions easier to see at a given video resolution [7].”
Template Matching Techniques in Computer Vision: Theory and Practice
Brunelli, R. (2009) · doi:10.1002/9780470744055 · Cited by 522
Sikuli: using GUI screenshots for search and automation
Yeh, T., Chang, T.-H., & Miller, R. C. (2009) · Proceedings of the 22nd annual ACM symposium on User interface software and technology · doi:10.1145/1622176.1622213 · Cited by 346 · cited 1×
“Methods that analyze user action traces [8,3,10] often capture application-level semantics about specific tools and their purpose, while techniques that analyze screen recordings [18,20,4] use computer vision to identify GUI interactions such as clicking on an icon or button.”
Stencils-based tutorials: design and evaluation
Kelleher, C., & Pausch, R. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055047 · Cited by 187
Chronicle: capture, exploration, and playback of document workflow histories
Grossman, T., Matejka, J., & Fitzmaurice, G. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866054 · Cited by 155 · cited 4×
“Methods that analyze user action traces [8,3,10] often capture application-level semantics about specific tools and their purpose, while techniques that analyze screen recordings [18,20,4] use computer vision to identify GUI interactions such as clicking on an icon or button.”
InfluentialPrefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure
Dixon, M., & Fogarty, J. A. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753554 · Cited by 150 · cited 1×
“Methods that analyze user action traces [8,3,10] often capture application-level semantics about specific tools and their purpose, while techniques that analyze screen recordings [18,20,4] use computer vision to identify GUI interactions such as clicking on an icon or button.”
Pause-and-play: automatically linking screencast video tutorials with applications
Pongnumkul, S., Dontcheva, M., Li, W., Wang, J., Bourdev, L., Avidan, S., & Cohen, M. F. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047213 · Cited by 144 · cited 6×
“Many efforts propose instructional aids that work in conjunction with the target application, including in-application step-by-step wizards [1,12,6] and Q&A forums [15], video-based tooltips [9], interactive video tutorials [18], command recommenders [14], and interface facades for mapping commands between application…”
InfluentialSelf-citationCommunityCommands: command recommendations for software applications
Matejka, J., Li, W., Grossman, T., & Fitzmaurice, G. (2009) · Proceedings of the 22nd annual ACM symposium on User interface software and technology · doi:10.1145/1622176.1622214 · Cited by 125 · cited 2×
“Many efforts propose instructional aids that work in conjunction with the target application, including in-application step-by-step wizards [1,12,6] and Q&A forums [15], video-based tooltips [9], interactive video tutorials [18], command recommenders [14], and interface facades for mapping commands between application…”
Self-citationToolClips: an investigation of contextual video assistance for functionality understanding
Grossman, T., & Fitzmaurice, G. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753552 · Cited by 125 · cited 3×
“studied the effectiveness of embedding short (10-25 second) video clips in applications [9].”
Cited by
Crowdsourcing step-by-step information extraction to enhance existing how-to videos
Kim, J., Nguyen, P. T., Weir, S., Guo, P. J., Miller, R. C., & Gajos, K. Z. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556986 · Cited by 130
AdapTutAR: An Adaptive Tutoring System for Machine Tasks in Augmented Reality
Huang, G., Qian, X., Wang, T., Patel, F., Sreeram, M., Cao, Y., Ramani, K., & Quinn, A. J. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445283 · Cited by 119
Learnersourcing Subgoal Labels for How-to Videos
Weir, S., Kim, J., Gajos, K. Z., & Miller, R. C. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675219 · Cited by 111
Chi, P.-Y., Liu, J., Linder, J., Dontcheva, M., Li, W., & Hartmann, B. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2502052 · Cited by 75
Self-citationSelective Undo Support for Painting Applications
Myers, B. A., Lai, A., Le, T. M., Yoon, Y., Faulring, A., & Brandt, J. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702543 · Cited by 73
Metadata
- Authors
- Chi, P.-Y., Ahn, S., Ren, A., Dontcheva, M., Li, W., & Hartmann, B.
- Affiliations
- University of California, Berkeley (US)
- Adobe Systems (United States) (US)
- Venue
- Proceedings of the 25th annual ACM symposium on User interface software and technology
- Publisher
- ACM
- Published
- 2012-10-07
- Pages
- 93-102
- OpenAlex
- W2029044878
- Cited by
- 113
- References
- 22
- Citations by year
- 2026: 6 · 2025: 13 · 2024: 13 · 2023: 15 · 2022: 6 · 2021: 7 · 2020: 6 · 2019: 3 · 2018: 7 · 2017: 3 · 2016: 3 · 2015: 12 · 2014: 12 · 2013: 7
- FWCI
- 6.97
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- software tutorial, video tutorials, video compositing, task completion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MixT: automatic generation of step-by-step mixed media tutorials. https://app.cochat.ai/papers/10.1145/2380116.2380130
- The paper
- Chi, P.-Y., Ahn, S., Ren, A., Dontcheva, M., Li, W., & Hartmann, B. (2012). MixT: automatic generation of step-by-step mixed media tutorials. Proceedings of the 25th annual ACM symposium on User interface software and technology. https://doi.org/10.1145/2380116.2380130
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.