"We Don't Do That Here": How Collaborative Editing with Mentors Improves Engagement in Social Q&A Communities
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Online question-and-answer (Q&A) communities like Stack Overflow have norms that are not obvious to novice users. Novices create and post programming questions without feedback, and the community enforces site norms through public downvoting and commenting. This can leave novices discouraged from further participation. We deployed a month long, just-in-time mentorship program to Stack Overflow in which we redirected novices in the process of asking a question to an on-site Help Room. There, novices received feedback on their question drafts from experienced Stack Overflow mentors. We present examples and discussion of various question improvements including: question context, code formatting, and wording that adheres to on-site cultural norms. We find that mentored questions are substantially improved over non-mentored questions, with average scores increasing by 50%. We provide design implications that challenge how socio-technical communities onboard novices across domains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Building Successful Online Communities: Evidence-Based Social Design
Kraut, R. E., Resnick, P., Kiesler, S., Burke, M., Chen, Y., Kittur, N., Konstan, J., Ren, Y., & Riedl, J. (2012) · doi:10.7551/mitpress/8472.001.0001 · Cited by 327
PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance
Kulkarni, C. E., Bernstein, M. S., & Klemmer, S. R. (2015) · Proceedings of the Second (2015) ACM Conference on Learning @ Scale · doi:10.1145/2724660.2724670 · Cited by 192
The Heart Work of Wikipedia: Gendered, Emotional Labor in the World's Largest Online Encyclopedia
Menking, A., & Erickson, I. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702514 · Cited by 107
Research works printed but not linked: 21
- 10.1016/j.jvb.2010.02.007 · DOI
- Muhammad Asaduzzaman, Ahmed Shah Mashiyat, Chanchal K. Roy, and Kevin A. Schneider. 2013. Answering Questions About Unanswered Questions of Stack Overflow. In Proceedings of the 10th Working Conference on Mining Software Repositories (MSR '13). 97--100.
- Peter Bailey, Nick Craswell, and David Hawking. 2000. Dark matter on the web. In Proceedings of the 2000 9th World Wide Web Conference. 2.
- 10.1007/BF03395423 · DOI · Paper page
- 10.1145/2998181.2998342 · DOI
- 10.1177/014920630102700105 · DOI
- 10.1145/2675133.2675254 · DOI
- 10.1145/2950290.2950331 · DOI · Paper page
- Lee Giles and Steve Lawrence. 1999. Accessibility and distribution of information on the web. Nature 400, 6740 (1999), 107--109.
- 10.1109/VLHCC.2015.7357201 · DOI
- Reid Hoffman. 2017. Masters of Scale: Handcrafted. (3 May 2017). Retrieved September 11, 2017 from https://mastersofscale.com/.
- 10.5555/2207798 · doi:10.5555/2207798
- Jean Lave and Etienne Wenger. 1991. Situated learning: Legitimate peripheral participation. Cambridge university press.
- 10.1145/1978942.1979366 · DOI · Paper page
- 10.1007/s10660-006-5985-x · DOI · Paper page
- 10.1145/2702123.2702294 · DOI
- 10.5555/2820518.2820586 · doi:10.5555/2820518.2820586
- 10.1145/2675133.2675215 · DOI · Paper page
- 10.1109/ICSE-SEET.2017.19 · DOI
- 10.1093/iwc/iwt047 · DOI · Paper page
- Etienne Wenger. 1998. Communities of practice: Learning, meaning, and identity. Cambridge university press.
Cited by
Generative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 311
How Do Users Revise Answers on Technical Q&A Websites? A Case Study on Stack Overflow
Wang, S., Chen, T.-H., & Hassan, A. E. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2874470 · Cited by 50
“For instance, answers may lack the explanations for some important concepts or references, which may in turn make such answers difficult to understand [1].”
BackgroundGender and Participation in Open Source Software Development
Frluckaj, H., Dabbish, L., Widder, D. G., Qiu, H. S., & Herbsleb, J. D. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555190 · Cited by 42
“For open source projects, our results suggest the potential for support through private mentorship akin to that explored by Ford et al. [36] to combat initial contribution barriers for women by providing a safe space for asking questions and a guardian to walk them through the contribution and…”
BackgroundCheng, R., Wang, R., Zimmermann, T., & Ford, D. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3651990 · Cited by 47
“Throughout all these forms of engagement, community members are still learning by observing experts [23] and often trying to connect experiences shared back to their own [22].”
BackgroundSelf-citationTen simple rules for helping newcomers become contributors to open projects
Sholler, D., Steinmacher, I., Ford, D., Averick, M., Hoye, M., & Wilson, G. (2019) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007296 · Cited by 35
Self-citationCritique Me: Exploring How Creators Publicly Request Feedback in an Online Critique Community
Cheng, R., Zeng, Z., Liu, M., & Dow, S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415232 · Cited by 33
“Studies on social Q&A sites show that questions edited by human mentors to fit community norms will result in more and higher quality answers [21].”
BackgroundUnderstanding How Programmers Can Use Annotations on Documentation
Horvath, A., Liu, M. X., Hendriksen, R., Shannon, C., Paterson, E., Jawad, K., Macvean, A., & Myers, B. A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502095 · Cited by 26
A disruptive research playbook for studying disruptive innovations
Storey, M.-A., Russo, D., Novielli, N., Kobayashi, T., & Wang, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3678172 · Cited by 15
Cheng, R., Dasgupta, S., & Hill, B. M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502124 · Cited by 15
“Additionally, the discussion forum could be seeded with prompts to support the identifcation of underlining conceptual knowledge and to explicitly connect examples with human mentoring [26], cognitive apprenticeship [37] and automatic annotation [12].”
BackgroundMarcu, G., & Huh-Yoo, J. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596009 · Cited by 11
Debugging with stack overflow: web search behavior in novice and expert programmers
Li, A., Endres, M., & Weimer, W. (2022) · Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training · doi:10.1145/3510456.3514147 · Cited by 8
Metadata
- Authors
- Ford, D., Lustig, K., Banks, J., & Parnin, C.
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-12
- OpenAlex
- W2796134555
- Cited by
- 55
- References
- 3
- FWCI
- 4.19
- Type
- conference paper
- Language
- en
- Open access
- bronze
- Funders
- National Science Foundation (DGE-1252376)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.