TMOSS: Using Intermediate Assignment Work to Understand Excessive Collaboration in Large Classes
Abstract
As computer science classes grow, instructor workload also increases: teachers must simultaneously teach material, provide assignment feedback, and monitor student progress. At scale, it is hard to know which students need extra help, and as a result some students can resort to excessive collaboration--using online resources or peer code--to complete their work. In this paper, we present TMOSS, a tool that analyzes the intermediate steps a student takes to complete a programming assignment. We find that for three separate course offerings, TMOSS is almost twice as effective as traditional software similarity detectors in identifying the number of students who exhibit excessive collaboration. We also find that such students spend significantly less time on their assignment, use fewer class tutoring resources, and perform worse on exams than their peers. Finally, we provide a theory of the parametric distribution of typical student assignment similarity, which allows for probabilistic interpretation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Schleimer, S., S, W. D., & Aiken, A. (2003) · Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data · doi:10.1145/872757.872770 · Cited by 1,099 · cited 7× · see citing works
“Many software similarity systems have been developed over the past two decades to detect for software plagiarism [7, 18, 20, 23, 24].”
BackgroundMethodInfluentialCheating among college and university students: A North American perspective
McCabe, D. L. (2005) · International Journal for Educational Integrity · doi:10.21913/ijei.v1i1.14 · Cited by 446 · see citing works
Wise, M. J. (1996) · ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/236452.236525 · Cited by 296 · see citing works
Les valeurs extrêmes des distributions statistiques
Gumbel, E. J. (1935) · French digital mathematics library (Numdam) · Cited by 225 · see citing works
Modeling how students learn to program
Piech, C., Sahami, M., Koller, D., Cooper, S., & Blikstein, P. (2012) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2157136.2157182 · Cited by 200 · see citing works
Self-citationTurning to Turnitin to Fight Plagiarism among University Students
Batane, T. (2010) · Cited by 182 · see citing works
Gitchell, D., & Trân, N. (1999) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/299649.299783 · Cited by 163 · see citing works
JPlag: finding plagiarisms among a set of programs
Prechelt, L., Malpohl, G., & Philippsen, M. (2000) · Karlsruhe · doi:10.5445/ir/542000 · Cited by 144 · see citing works
MOSS : A system for detecting software plagiarism
Aiken, A. (2004) · Medical Entomology and Zoology · Cited by 123 · see citing works
Revisiting why students drop CS1
Petersen, A., Craig, M., Campbell, J., & Tafliovich, A. (2016) · Koli Calling International Conference on Computing Education Research · doi:10.1145/2999541.2999552 · Cited by 101 · see citing works
Using undergraduates as teaching assistants in introductory programming courses
Roberts, E. S., Lilly, J. F., & Rollins, B. (1995) · ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/199688.199716 · Cited by 70 · see citing works
Parlante, N., Wolfman, S. A., McCann, L. I., Roberts, E. S., Nevison, C., Motil, J., Cain, J., & Reges, S. (2006) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/1121341.1121516 · Cited by 45 · see citing works
Investigating Student Plagiarism Patterns and Correlations to Grades
Pierce, J., & Zilles, C. B. (2017) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3017680.3017797 · Cited by 41 · see citing works
Enhanced User Authentication Through Keystroke Biometrics
Lau, E. Y., Liu, X., Chen, X., & Yu, X. F. (2004) · Cited by 41 · see citing works
Proceedings of the 18th European Conference on e-Learning
(2019) · doi:10.34190/eel.19 · Cited by 41 · see citing works
Scaling Introductory Courses Using Undergraduate Teaching Assistants
Forbes, J., Malan, D. J., Pon-Barry, H., Reges, S., & Sahami, M. (2017) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3017680.3017694 · Cited by 39 · see citing works
Self-citationMahdi, S., & Cenac, M. (2005) · Revista de Matemática Teoría y Aplicaciones · doi:10.15517/rmta.v12i1-2.259 · Cited by 36 · see citing works
Offering Verified Credentials in Massive Open Online Courses
Maas, A. L., Heather, C., Do, C. (., Brandman, R., Koller, D., & Ng, A. Y. (2014) · Ubiquity · doi:10.1145/2591684 · Cited by 35 · see citing works
Detecting Plagiarism Based on the Creation Process
Schneider, J., Bernstein, A., Brocke, J. V., Damevski, K., & Shepherd, D. C. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2720171 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Plagiarism in Programming Assessments: A Systematic Review
Albluwi, I. (2020) · ACM Transactions on Computing Education · doi:10.1145/3371156 · Cited by 63 · see citing works
Collaboration Versus Cheating: Reducing Code Plagiarism in an Online MS Computer Science Program
Mason, T., Gavrilovska, A., & Joyner, D. A. (2019) · Proceedings of the 50th ACM Technical Symposium on Computer Science Education · doi:10.1145/3287324.3287443 · Cited by 39 · see citing works
Mossad: defeating software plagiarism detection
Devore-McDonald, B., & Berger, E. D. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428206 · Cited by 35 · see citing works
Yan, L., Hu, A., & Piech, C. (2019) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3287324.3287483 · Cited by 32 · see citing works
Self-citationRusak, G., & Yan, L. (2021) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3408877.3432556 · Cited by 28 · see citing works
Self-citationChoosing Code Segments to Exclude from Code Similarity Detection
Simon, Karnalim, O., Sheard, J., Dema, I., Karkare, A., Leinonen, J., Liut, M., & McCauley, R. A. (2020) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3437800.3439201 · Cited by 25 · see citing works
Source Code Plagiarism Detection in an Educational Context: A Literature Mapping
Aniceto, R. C., de Holanda, M. T., Castanho, C. D., & Silva, D. D. (2021) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie49875.2021.9637155 · Cited by 22 · see citing works
Silva, D. B., Aguiar, R. D. L., Dvconlo, D. S., & Junior, C. N. S. (2019) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie43999.2019.9028702 · Cited by 22 · see citing works
Automated Detection of AI-Obfuscated Plagiarism in Modeling Assignments
Sağlam, T., Hahner, S., Schmid, L., & Burger, E. (2024) · Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training · doi:10.1145/3639474.3640084 · Cited by 17 · see citing works
Cheating Detection in Online Assessments via Timeline Analysis
Du, J., Song, Y., An, M., An, M., Bogart, C., & Sakr, M. F. (2022) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3478431.3499368 · Cited by 14 · see citing works
Collaboration Versus Cheating: Reducing Code Plagiarism in an Online MS Computer Science ProgramPreprint
Mason, T., Gavrilovska, A., & Joyner, D. A. (2019) · arXiv (Cornell University) · Cited by 14 · see citing works
Detecting Automatic Software Plagiarism via Token Sequence Normalization
Sağlam, T., Brödel, M., Schmid, L., & Hahner, S. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639192 · Cited by 9 · see citing works
Layered similarity detection for programming plagiarism and collusion on weekly assessments
Karnalim, O., Simon, N. P., & Chivers, W. (2022) · Computer Applications in Engineering Education · doi:10.1002/cae.22553 · Cited by 9 · see citing works
Yan, L., McKeown, N., & Piech, C. (2019) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3287324.3287386 · Cited by 8 · see citing works
Self-citationKeystroke Data in Programming Courses
Leinonen, J. (2019) · Työväentutkimus Vuosikirja · Cited by 8 · see citing works
Barros, B., Conejo, R., Ruiz-Sepúlveda, A., & Triguero‐Ruiz, F. (2021) · Applied Sciences · doi:10.3390/app11199328 · Cited by 7 · see citing works
Using early plagiarism detection in programming classes to address the student’s difficulties
Fonseca, N. G., Macedo, L., & Mendes, A. J. (2018) · International Symposium on Computers in Education · doi:10.1109/siie.2018.8586700 · Cited by 6 · see citing works
Metrics for Student Classroom Engagement and Correlation to Software Assignment Plagiarism
Allen, W. H., Belsky, S., Kelly, B., Barela, J., Peveler, M., & Cutler, B. M. (2022) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3478432.3499133 · Cited by 3 · see citing works
Bãutu, E., & Bãutu, A. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-94120-2_48 · Cited by 3 · see citing works
Kassil, A., & Ball, M. (2021) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3408877.3439674 · Cited by 2 · see citing works
Wortzman, B., Stephens-Martinez, K., Minnes, M., Ola, O., & Blank, A. (2022) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3545947.3569609 · Cited by 2 · see citing works
A Fast and Accurate Machine Learning Autograder for the Breakout Assignment
Liu, E. Z., Yuan, D. D., Ahmed, A., Cornwall, E., Woodrow, J., Burns, K., Nie, A., Brunskill, E., Piech, C., & Finn, C. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630759 · Cited by 2 · see citing works
Self-citationMidterm Exam Outliers Efficiently Highlight Potential Cheaters on Programming Assignments
Shirazi, S. H. A., Pang, A., Knight, A., Salloum, M., & Vahid, F. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701883 · Cited by 2 · see citing works
Maisch, R., Hagel, N., & Bartel, A. (2025) · European Conference on Software Engineering Education (ECSEE) · doi:10.1145/3723010.3723019 · Cited by 1 · see citing works
Machine Learning Models to Detect AI-Assisted Code Anomaly in Introductory Programming Course
Toba, H., & Karnalim, O. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-71530-3_11 · Cited by 1 · see citing works
A Berkeley View of Teaching CS at ScalePreprint
Lin, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.07081 · Cited by 1 · see citing works
Unique Exams: Designing assessments for integrity and fairnessPreprint
Rusak, G., & Yan, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01713 · Cited by 1 · see citing works
Self-citationCriterion-Based Grading, Agile Goal Setting, and Course (Un)Completion Strategies
Ihantola, P., Isohanni, E., Heino, P., & Mikkonen, T. (2018) · Springer eBooks · doi:10.1007/978-981-13-2751-3_11
Sensitive Similarity on Programming Assessments Expecting Highly Similar Submissions
Karnalim, O. (2024) · IEEE World Engineering Education Conference (EDUNINE) · doi:10.1109/edunine60625.2024.10500603
Revisiting Plagiarism Deterrence in CS1 Through Keystroke Data
Syndergaard, C., & Edwards, J. (2024) · Intermountain Engineering, Technology and Computing (IETC) · doi:10.1109/ietc61393.2024.10564457
Mitigating Obfuscation Attacks on Software Plagiarism Detectors via Subsequence Merging
Sağlam, T., Niehues, N., Hahner, S., & Schmid, L. (2025) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet66350.2025.00028
- Authors
- Yan, L., McKeown, N., Sahami, M., & Piech, C.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 49th ACM Technical Symposium on Computer Science Education
- Publisher
- ACM
- Published
- 2018-02-21
- Pages
- 110-115
- OpenAlex
- W2789176835
- Cited by
- 30
- References
- 19
- Citations by year
- 2025: 3 · 2024: 6 · 2022: 4 · 2021: 4 · 2020: 3 · 2019: 7 · 2018: 3
- FWCI
- 7.60
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion)
- Funders
- National Science Foundation (http://dx.doi.org/10.13039/100000001) (DGE-114747)
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- online resources, programming exercises, student performance, instructor workload, programming education, parametric distributions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review