Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 546

The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.

A Survey of Automated Assessment Approaches for Programming Assignments

Ala-Mutka, Kirsti M · 2005 · Computer Science Education

Abstract

Practical programming is one of the basic skills pursued in computer science education. On programming courses, the coursework consists of programming assignments that need to be assessed from different points of view. Since the submitted assignments are executable programs with a formal structure, some features can be assessed automatically. The basic requirement for automated assessment is the numerical measurability of assessment targets, but semiautomatic approaches can overcome this restriction. Recognizing automatically assessable features can help teachers to create educational models, where automatic tools let teachers concentrate their work on the learning issues that need student-teacher interaction the most. Several automatic tools for both static and dynamic assessment of computer programs have been reported in the literature. This article promotes these issues by surveying several automatic approaches for assessing programming assignments. Not all the existing tools will be covered, simply because of the vast number of them. The article concentrates on bringing forward different assessment techniques and approaches to give an interested reader starting points for finding further information in the area. Automatic assessment tools can be used to help teachers in grading tasks as well as to support students' working process with automatic feedback. Common advantages of automation are the speed, availability, consistency and objectivity of assessment. However, automatic tools emphasize the need for careful pedagogical design of the assignment and assessment settings. To effectively share the knowledge and good assessment solutions already developed, better interoperability and portability of the tools is needed.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/08993400500150747

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 26

  1. Arnow D, 1999, Proceedings of the 29th ASEE/IEEE Frontiers in Education Conference, 12A9/39 – 12A9/
  2. Bettini L Crescenzi P Innocenti G Loreti M Cecchi L 2004 An Environment for Self-Assessing Java Programming Skills in Undergraduate First Programming Courses In Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies Finland, 161 – 165 · doi:10.1109/icalt.2004.1357395
  3. 10.1109/TE.2004.825220 · doi:10.1109/te.2004.825220
  4. 10.1016/S0360-1315(97)00054-7 · doi:10.1016/s0360-1315(97)00054-7
  5. Ellsworth C Fenwick J Kurtz B 2004 The Quiver System In Proceedings of the 35th SIGCSE technical symposium on Computer Science Education US, 205 – 209 · doi:10.1145/971300.971374
  6. English J 2004 Automatic Assessment of GUI Programs using JEWL In Proceedings of 9th annual conference on Innovation and technology in computer science education UK, 137 – 141 · doi:10.1145/1007996.1008033
  7. English J 2002 Experience with a computer-assisted formal programming examination In Proceedings of 7th annual conference on Innovation and technology in computer science education Denmark, 51 – 54 · doi:10.1145/544414.544432
  8. Hansen H Ruuska M 2003 Assessing time-efficiency in a course on data structures and algorithms In Proceedings of the 3rd Annual Finnish/Baltic Sea Conference on Computer Science Education , Finland
  9. 10.1023/A:1026364126982 · doi:10.1023/a:1026364126982
  10. Higgins C Symeonidis P Tsinsifas A 2002 Diagram-based CBA using DATsys and CourseMaster In Proceedings of the International Conference on Computers in Education 367 372
  11. 10.1145/782941.782978 · doi:10.1145/782941.782978
  12. 10.1016/0360-1315(93)90086-X · DOI
  13. Joy M Griffiths N 2004 Online Submission of Coursework – a Technological Perspective In Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies Finland, 430 – 434 · doi:10.1109/icalt.2004.1357451
  14. Kay D Scott T Isaacson P Reek K 1994 Automated grading assistance for student programs In Proceedings of the 25th SIGCSE technical symposium on Computer science education USA, 381 – 382 · doi:10.1145/191033.191184
  15. Lister R Leaney J 2003 First Year Programming: Let All the Flowers Bloom In T. Greening & R. Lister (Eds.), Computing Education 2003 Fifth Australasian Computing Education Conference (pp. 221 – 230)
  16. 10.1002/(SICI)1097-024X(19990710)29:8<721::AID-SPE257>3.0.CO;2-0 · DOI
  17. Mengel SA Ulans JV 1999 A case study of the analysis of the quality of novice students programs In Proceedings of the 12th Conference on Software Engineering Education and Training 40 49 · doi:10.1109/csee.1999.755178
  18. Michaelson G 1996 Automatic Analysis of Functional Program Style In Proceedings of Australian Software Engineering Conference 38 46 · doi:10.1109/aswec.1996.534121
  19. MOSS. [Computer software] Retrieved November 15, 2004, from http://www.cs.berkeley.edu/∼aiken/moss.html · doi:10.1016/b978-075067740-0/50002-0
  20. Olson DM 1988 The reliability of analytic and holistic methods in rating students' computer programs In Proceedings of the 19th SIGCSE technical symposium on Computer science education USA, 293 – 298 · doi:10.1145/52965.53037
  21. 10.1145/857076.857090 · DOI
  22. Reek KA 1989 The TRY system -or- how to avoid testing student programs In Proceedings of the 20th SIGCSE technical symposium on Computer science education USA, 112 – 116 · doi:10.1145/65294.71198
  23. Ruehr F, 2002, Journal of Computing in Small Colleges, 18, 65
  24. Truong N Roe P Bancroft P 2003 A Web Based Environment for Learning to Program In Proceedings of the 25th Australasian Computer Science Conference Australia, 255 – 264
  25. Xie Y Engler D 2002 Using Redundancies to Find Errors In Proceedings of SIGSOFT 2002/FSE-10 USA, 51 – 60 · doi:10.1145/587051.587060
  26. Zeller A 2000 Making students read and review code In Proceedings of 5th annual conference on Innovation and technology in computer science education Finland, 89 – 92 · doi:10.1145/343048.343090

Cited by

See all 546

Metadata

Authors
Ala-Mutka, K. M.
Venue
Computer Science Education
Published
2005-06
Volume
15
Issue
2
Pages
83-102
ISSN
  • 0899-3408 (ISSN-L)
  • 1744-5175
OpenAlex
W2070405152
Cited by
546
References
39
FWCI
18.22
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.