Measuring programming experience
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Programming experience is an important confounding parameter in controlled experiments regarding program comprehension. In literature, ways to measure or control programming experience vary. Often, researchers neglect it or do not specify how they controlled it. We set out to find a well-defined understanding of programming experience and a way to measure it. From published comprehension experiments, we extracted questions that assess programming experience. In a controlled experiment, we compare the answers of 128 students to these questions with their performance in solving program-comprehension tasks. We found that self estimation seems to be a reliable way to measure programming experience. Furthermore, we applied exploratory factor analysis to extract a model of programming experience. With our analysis, we initiate a path toward measuring programming experience with a valid and reliable tool, so that we can control its influence on program comprehension.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
Costello, A. B., & Osborne, J. (2005) · University of Massachusetts Amherst · doi:10.7275/jyj1-4868 · Cited by 10,297
Practical issues in structural modeling
BENTLER, P. M., & CHOU, C.-P. (1987) · Sociological Methods & Research · doi:10.1177/0049124187016001004 · Cited by 5,804
Are Reviews an Alternative to Pair Programming?
Müller, M. M. (2004) · Empirical Software Engineering · doi:10.1023/b:emse.0000039883.47173.39 · Cited by 61 · cited 1×
“For example, Müller [23] asked how many lines of code the largest program has that subjects have implemented.”
Background
Cited by
An Empirical Investigation into Programming Language Syntax
Stefik, A., & Siebert, S. (2013) · ACM Transactions on Computing Education · doi:10.1145/2534973 · Cited by 295
“…if they self-reported any non-zero amount of total programming experience, self-reporting, or how individuals are classified, were the most appropriate, is not clear, but Feigenspan et al. published results from a survey of programming experience with similarities to ours [Feigenspan et al. 2012].”
Result(2015) · doi:10.1007/s10664-013-9279-3 · Cited by 273
Understanding understanding source code with functional magnetic resonance imaging
Siegmund, J., Kästner, C., Apel, S., Parnin, C., Bethmann, A., Leich, T., Saake, G., & Brechmann, A. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568252 · Cited by 219
“Furthermore, we assessed the programming experience of participants with an empirically developed questionnaire to assure a homogeneous level of programming experience [23], and we assessed the handedness of our participants with the Edinburgh Handedness Inventory [49], because the handedness correlates with the role…”
MethodSelf-citation(2022) · doi:10.1145/3491102.3501957 · Cited by 47
A family of experiments on test-driven development
Santos, A., Vegas, S., Dieste, O., Uyaguari, F., Tosun, A., Fucci, D., Turhan, B., Scanniello, G., Romano, S., Karac, I., et al. (2021) · Empirical Software Engineering · doi:10.1007/s10664-020-09895-8 · Cited by 30
“In view of this, and as no consensus seems to have been reached yet on how best to measure developer experience in SE [7,16,17], we decided to remove the time periods that we had allotted to each experience level.”
BackgroundMethodResultInfluential
Metadata
- Authors
- Feigenspan, J., Kästner, C., Liebig, J., Apel, S., & Hanenberg, S.
- Venue
- 2012 20th IEEE International Conference on Program Comprehension (ICPC)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 73-82
- OpenAlex
- W2092349198
- Cited by
- 106
- References
- 4
- FWCI
- 5.75
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.