Academic Source Code Plagiarism Detection by Measuring Program Behavioral Similarity

All source code plagiarism detection tools that are known to be available use only structural measures of similarity.

“Furthermore, all known available SCPDTs implement structural measures of similarity only.”

• cites [8] 8. V. T. Martins, D. Fonte, P. R. Henriques, and D. da Cruz, ''Plagiarism detection: A tool survey and comparison,'' in Proc. 3rd Symp. Lang., Appl. Technol., in OpenAccess Series in Informatics, vol. 38, M. J. V. Pereira, J. P. Leal, and A. Simões, Eds. Dagstuhl, Germany: Schloss Dagstuhl- Leibniz-Zentrum fuer Informatik, 2014, pp. 143-158. sourceDisputed

  Section 2.2, “Approaches” (page unavailable), distinguishes attribute-based, token-based, and structure-based methods rather than saying all tools use structural measures only. It describes token-based tools that add some structure-dependent modifications; Section 4.4 identifies Marble as using token comparison. These passages contradict the claim’s “all” and “only.” (LLM, conf=0.90)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [9] 9. M. Novak, M. Joy, and D. Kermek, ''Source-code similarity detection and detection tools used in academia: A systematic review,'' ACM Trans. Comput. Educ., vol. 19, no. 3, pp. 1-37, Jun. 2019. sourceNot checked — no full text of the source

Every source code plagiarism detection tool proposed so far uses structural or semantic measures of similarity.

“All known proposed SCPDTs implement structural or semantic measures of similarity.”

• cites [8] 8. V. T. Martins, D. Fonte, P. R. Henriques, and D. da Cruz, ''Plagiarism detection: A tool survey and comparison,'' in Proc. 3rd Symp. Lang., Appl. Technol., in OpenAccess Series in Informatics, vol. 38, M. J. V. Pereira, J. P. Leal, and A. Simões, Eds. Dagstuhl, Germany: Schloss Dagstuhl- Leibniz-Zentrum fuer Informatik, 2014, pp. 143-158. sourceDisputed

  Section 2.2 (page not supplied) describes attribute-based and token-based methods as well as structure-based methods. Its examples include comparisons using code size and token hashes, contradicting the claim that all proposed tools use structural or semantic similarity measures. (LLM, conf=0.87)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [9] 9. M. Novak, M. Joy, and D. Kermek, ''Source-code similarity detection and detection tools used in academia: A systematic review,'' ACM Trans. Comput. Educ., vol. 19, no. 3, pp. 1-37, Jun. 2019. sourceNot checked — no full text of the source

BPlag is more accurate at detecting plagiarised source code than the five compared source code plagiarism detection tools.

“It is then shown that BPlag is more robust to plagiarism-hiding transformations and more accurate in the detection of plagiarised source code, but is less efficient than the compared tools.”

BPlag is less efficient than the five compared source code plagiarism detection tools.

“It is then shown that BPlag is more robust to plagiarism-hiding transformations and more accurate in the detection of plagiarised source code, but is less efficient than the compared tools.”

BPlag is more robust to plagiarism-hiding transformations than the five compared source code plagiarism detection tools.

“It is then shown that BPlag is more robust to plagiarism-hiding transformations and more accurate in the detection of plagiarised source code, but is less efficient than the compared tools.”

Starting with L3 transformations, tools suffer a sharp and noticeable decrease in measured similarity.

“However, starting with the application of L3 transformations, the tools suffer from a sharp and noticeable decrease in measured similarity.”

At 20% transformation chance, BPlag is slightly more robust than all other tools.

“At this transformation chance, there is a negligible decrease in similarity, however, BPlag is slightly more robust than all other tools.”

At L3, certain transformations have a large impact on BPlag's similarity evaluation.

“This implies that at L3, certain transformations have a large impact on the similarity of the variants as evaluated by BPlag.”

As transformation chance increases, the average drop in similarity increases for each level of transformation.

“Firstly, as the chance of transformation increases, the average drop in similarity increases for each level of transformation.”

All tools show a sudden increase in average drop in similarity for variants constructed with L3 transformations.

“Secondly, all tools begin to show a sudden increase in the average drop in similarity for variants constructed with L3 transformations.”

For the five compared SCPDTs, the decrease in similarity continues at L4 and L5 transformations.

“For the five compared SCPDTs, this decrease continues at the L4 and L5 transformations.”

For all tools except BPlag, the drop in similarity continues with L4 and L5 transformations.

“For all tools but BPlag, this drop continues with the L4 and L5 transformations.”

At 40% transformation chance, for L1 and L2 variants, all tools except Sherlock-S and Sherlock-W demonstrate negligible decreases in similarity.

“For variants created with the L1 and L2 transformations, all tools (excluding Sherlock-S and Sherlock-W) demonstrate negligible decreases in similarity.”

Transformations applied to L4 and L5 variants do not impact BPlag.

“However, the transformations applied to the L4 and L5 variants do not impact upon BPlag.”

BPlag is vulnerable to value-injecting transformations.

“However, as identified, BPlag is vulnerable to value-injecting transformations.”

Both transformations add new values represented in PIDGs.

“Both of these transformations result in the addition of new values represented in PIDGs.”

For variants made with L3 transformations, all tools record a much larger average drop in similarity.

“However, for variants created with the L3 transformations, a much larger average drop in similarity is recorded for all tools.”

BPlag is vulnerable to specific L3 transformations.

“However, from the results, BPlag is vulnerable to specific L3 transformations.”

At the 60% transformation chance, BPlag shows greater robustness to transformation.

“The 60% transformation chance provides clear evidence that BPlag has greater robustness to transformation.”

As transformation chance increases, BPlag demonstrates progressively greater robustness, indicated by lower average similarity drop.

“As the chance of transformation increases, BPlag demonstrates progressively greater robustness to transformation, indicated by the lower average drop in similarity.”

In this experiment, the noise appears as small value additions to PIDGs that cannot be mapped between two programs.

“In this experiment, the noise is manifested as small value additions to the PIDGs, that cannot otherwise be mapped between two programs.”

Variants created with value-injecting transformations have a much larger average drop in similarity than variants without them.

“It can be seen that there is a much larger average drop in similarity for variants created with value-injecting transformations, compared to those that do not contain valueinjecting transformations.”

The 12 assignment collections contained known cases of academic misconduct.

“Amongst the 12 collections of assignments, there are known cases of academic misconduct present.”

Without value-injecting transformations, BPlag demonstrates much greater robustness to transformation.

“Without valueinjecting transformations, BPlag demonstrates much greater robustness to transformation.”

The five compared tools show greater vulnerability to transformation at all levels and all transformation chances.

“The five compared tools show greater vulnerability to transformation at all levels of transformation, with all transformation chances.”

BPlag consistently ranks lowest in average drop in similarity.

“This is opposed to BPlag that consistently ranks lowest in terms of the average drop in similarity.”

Under the revised method, the average drop in similarity against L1 transformations is larger than against L2 transformations.

“For example, the average drop in similarity against L1 transformations is larger than the L2 transformations.”

For L3+ variants, there is a substantial reduction in similarity drop for variants created without value-injecting transformations.

“In all cases, for L3+ variants there is a substantial reduction in the drop of similarity for variants created without value-injecting transformations.”

BPlag shows a sharp decrease in robustness to L3 transformations.

“However, BPlag shows a sharp decrease in robustness to L3 transformations.”

The drop in similarity against L3 transformations is largely caused by two source transformations: declare redundant constants and assign default value to variable declaration.

“The drop in similarity against L3 transformations is largely caused by two specific source transformations: 'declare redundant constants', and 'assign default value to variable declaration'.”

For variants made with L1 and L2 transformations, all tools except Sherlock-S and Sherlock-W show negligible decreases in similarity.

“Initially, for variants created with L1 and L2 transformations, all tools (excluding Sherlock-S and Sherlock-W) again show negligible decreases in similarity.”

In pilot evaluations, the heuristic's limitations (incorrect initial mapping, disconnected components without operators/API calls) have minimal impact on similarity evaluation.

“However, in pilot evaluations this has minimal impact on the evaluation of similarity.”

BPlag is much more accurate in the presence of pervasive plagiarism-hiding transformations.

“From the results, BPlag is much more accurate in the presence of pervasive plagiarism-hiding transformations.”

This results in a lower error count at all levels of transformation and transformation chance combinations.

“Overall, this has resulted in a lower error count at all levels of transformation and transformation chance combinations.”

BPlag's results at the 100% transformation chance are more similar to the five compared tools' results at the 40% or 60% transformation chances.

“However, the results of BPlag at the 100% transformation chance are more similar to the five compared tools results at the 40% or 60% transformation chances.”

For a more reasonably sized data set of 100 submissions, BPlag would take approximately 45 minutes on an 8-core machine.

“Furthermore, if a more reasonably sized data set was analysed, (e.g. 100 submissions) it would take approximately 45 mins on an 8-core machine.”

Sim, JPlag, and Sherlock-W shuffle between second, third, and fourth place.

“This is followed by Sim, JPlag, and Sherlock-W that shuffle between second, third and fourth place.”

For L3+ transformations applied with higher transformation chances, this appears to have a significant impact on accuracy.

“For L3+ transformations applied with higher transformation chances (i.e. with transformations that are more pervasively applied), this appears to have a significant impact on accuracy.”

Including the modifications to accommodate value-injecting transformations significantly decreases BPlag's error counts at all transformation chances.

“However, by including the modifications to accommodate for value-injecting transformations, the error counts of BPlag are significantly decreased at all transformation chances, as shown in table 9.”

BPlag is subject to a progressively smaller decrease in accuracy when exposed to L3+ transformations.

“Comparatively, BPlag is subject to a progressively smaller decrease in accuracy when exposed to L3+ transformations.”

The compared tools saw progressively larger decreases in accuracy against L3+ transformations.

“Furthermore, the compared tools saw progressively larger decreases of accuracy against the L3+ transformations.”

BPlag commits the fewest errors when detecting the simulated plagiarised variants.

“Hence, it commits the fewest errors when detecting the simulated plagiarised variants.”

The evaluations rely on generated test data created with SPPlagiarise.

“The conducted evaluations rely on generated test data created with SPPlagiarise.”

BPlag is not the most efficient SCPDT.

“From these results, BPlag is not the most efficient SCPDT.”

At 60%, 80%, and 100% transformation chances, error counts of all tools, including BPlag, continue to increase.

“At the 60%, 80% and 100% transformation chances, the error counts of all tools, including BPlag, continue to increase.”

The SCPDTs generally see a small decrease in similarity against L1 and L2 transformations.

“The SCPDTs will generally see a small decrease in similarity against L1 and L2 transformations.”

At the 40% transformation chance, BPlag has cumulatively 17 errors over the five transformation levels.

“For example, at the 40% transformation chance, BPlag has cumulatively 17 errors over the five levels of transformation.”

At the 40% transformation chance, the next closest tool, Sim, has cumulatively 46 errors.

“This is compared to the next closest tool, Sim, that has cumulatively 46 errors recorded.”

BPlag experiences the least increase in errors among all tools.

“However, BPlag experiences the least increase in errors out of all the tools.”

On a modern 8-core machine, BPlag would take approximately 3 hours on collection 1.

“On a modern 8-core machine, BPlag would take approximately 3 hrs to execute on collection 1.”

BPlag is the most accurate in detecting simulated plagiarism across the six compared SCPDTs.

“From the results in table 8, BPlag is shown to be the most accurate in detecting the simulated cases of plagiarism across the six compared SCPDTs.”

BPlag shows vulnerability to transformations that modify program behaviour.

“However, the approach does show vulnerability to transformations that modify the behaviour of a program.”

The revised approach produces an immediately noticeable decrease in error counts at all transformation chances.

“Overall, there is an immediately noticeable decrease in the error counts at all transformation chances.”

All errors in this experiment result from submission pairs being incorrectly identified as plagiarised or innocent.

“All errors in this experiment result from submission pairs being incorrectly identified as plagiarised (for programs originally contained in the 12 data sets) or innocent (for the generated variants).”

On the largest data set (collection 1, 170 programs), BPlag would take approximately 24 hours single-threaded.

“If BPlag was running on the largest data set (collection 1, 170 programs), it would take approximately 24 hrs to run single-threaded.”

BPlag suffers a decrease in accuracy when detecting L3+ variants.

“However, BPlag does suffer a decrease in accuracy when detecting the L3+ variants.”

The efficiency order is Sherlock-S, then Sim, Plaggie, JPlag, Sherlock-W, and finally BPlag.

“This is followed by Sim, Plaggie, JPlag, Sherlock-W, and finally, BPlag.”

Sherlock-S is by far the most efficient tool on average.

“It can be seen that Sherlock-S is by far the most efficient tool on average.”

The most profound decrease is 230 errors at the 100% transformation chance.

“Most profound is the decrease of 230 errors at the 100% transformation chance.”

When tools are ranked by lowest cumulative errors, BPlag is consistently first.

“If the tools are ranked by lowest cumulative errors, BPlag is consistently first.”

The evaluations showed that BPlag is relatively less efficient than the tools it was compared with.

“However, the evaluations have also shown that BPlag is comparatively less efficient.”

In the evaluation, the existing SCPDTs that measure structural similarity were affected by every transformation, because the transformations alter source code structure.

“This was shown in the evaluation where the existing SCPDTs that measure structural similarity of source code are impacted by all transformations (as the transformations modify the source code structure).”

If a detection tool is not robust and examines features altered by plagiarism-hiding transformations, it may assign too little similarity and produce false negatives.

“Conversely, if a SCPDT is not robust to plagiarism-hiding transformations, and analyses aspects of source code that change due to plagiarismhiding transformations, a SCPDT may become too reserved in measuring similarity.This can result in the tool failing to measure a high enough similarity to indicate plagiarism (a false negative).”

A source code plagiarism detection tool can become robust to plagiarism-hiding transformations by disregarding source code features that plagiarisers often change.

“A SCPDT can gain robustness to plagiarism-hiding transformations by ignoring aspects of source code that plagiarisers commonly modify (e.g.ignoring comments or identifiers in source code [11]).”

• cites [11] 11. L. Prechelt, G. Malpohl, and M. Philippsen, ''Finding plagiarisms among a set of programs with JPlag,'' J-JUCS, vol. 8, no. 11, pp. 1016-1038, Nov. 2002.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

If a detection tool disregards too many program features, it may assign higher similarity to unrelated works and produce false positives.

“However, if a SCPDT ignores too many aspects of a program, the SCPDT may measure greater similarity between unrelated works.This can result in the SCPDT being inaccurate by indicating unrelated works as being plagiarised (a false positive).”

Pervasive plagiarism-hiding transformations can severely impair a detection tool's ability to measure similarity.

“Such transformations can greatly impact upon the ability of SCPDTs to measure similarity.”

• cites [12] 12. H. Cheers, Y. Lin, and S. P. Smith, ''Detecting pervasive source code plagiarism through dynamic program behaviours,'' in Proc. 22nd Aus- tralas. Comput. Educ. Conf. (ACE). New York, NY, USA: Association for Computing Machinery, 2020, pp. 21-30. sourceNot checked — no full text of the source

Plagiarism-hiding transformations are usually cosmetic: they alter source code appearance and structure while preserving the original program's behaviour.

“Such source code transformations are typically cosmetic in nature, in that they change the appearance and structure of the source code [6], [7], [10], but retain the original behaviour of the plagiarised assignment.”

• cites [6] 6. A. Parker and J. O. Hamblen, ''Computer algorithms for plagiarism detec- tion,'' IEEE Trans. Educ., vol. 32, no. 2, pp. 94-99, May 1989. sourceNot checked — no full text of the source
• cites [7] 7. M. Joy and M. Luck, ''Plagiarism in programming assignments,'' IEEE Trans. Educ., vol. 42, no. 2, pp. 129-133, May 1999. sourceNot checked — no full text of the source
• cites [10] 10. J. A. W. Faidhi and S. K. Robinson, ''An empirical approach for detecting program similarity and plagiarism within a university programming envi- ronment,'' Comput. Educ., vol. 11, no. 1, pp. 11-19, Jan. 1987. sourceNot checked — no full text of the source

To change execution behaviour while retaining input-output relations, a plagiariser would need a different data model, a different data transformation process, and different execution environment interactions.

“This means a plagiariser would have to transform the implementation of a programs execution behaviour such that it uses a different data model, different process of transforming data, and different interactions with the execution environment; while retaining the original input-output relations.”

With symbolic execution, BPlag can analyse any valid Java program without user-provided test inputs.

“Hence, with symbolic execution, BPlag can analyse any valid Java program without user-provided test inputs.”

Functional equivalence is unsuitable for source code plagiarism detection because it would treat all correctly implemented submissions as identical.

“Therefore, comparing the behaviour of a program with functional equivalence is not suitable in source code plagiarism detection, as it would result in all correctly implemented submissions being detected as identical.”

When a plagiarised work has been pervasively transformed, detection tools can report similarity too low to raise suspicion.

“Subsequently, when a plagiarised work is pervasively transformed, SCPDTs can report a low enough similarity that it does not raise suspicion of plagiarism [12].”

• cites [12] 12. H. Cheers, Y. Lin, and S. P. Smith, ''Detecting pervasive source code plagiarism through dynamic program behaviours,'' in Proc. 22nd Aus- tralas. Comput. Educ. Conf. (ACE). New York, NY, USA: Association for Computing Machinery, 2020, pp. 21-30. sourceNot checked — no full text of the source

Typically, the original behaviour of a plagiarised program remains largely unchanged.

“Hence, typically the original behaviour of a plagiarised program remains largely unchanged.”

BPlag's behavioural approach can distinguish programs that are behaviourally similar because they implement the same specification from those that are behaviourally similar because of plagiarism, in a robust and accurate way.

“Subsequently, this allows for a behavioural approach to SCPD that can differentiate between programs that are behaviourally similar due to implementing the same specification, and those that are behaviourally similar due to plagiarism; in a manner that is both robust and accurate.”

The three-aspect model gives only an approximate representation of a program's execution behaviour.

“However, this model can only be termed to provide an 'approximate' representation of the execution behaviour of a program.”

Structural transformations correspond to L3 and higher transformations in Faidhi and Robinson's taxonomy.

“Structural transformations are equivalent to L3+ transformations from Faidhi and Robinson's [10] taxonomy.”

• cites [10] 10. J. A. W. Faidhi and S. K. Robinson, ''An empirical approach for detecting program similarity and plagiarism within a university programming envi- ronment,'' Comput. Educ., vol. 11, no. 1, pp. 11-19, Jan. 1987. sourceNot checked — no full text of the source

Most plagiarism-hiding transformations leave a program's execution behaviour unchanged.

“The majority of these transformations do not modify the execution behaviour of a program.This is as a plagiariser is commonly somebody who is either: inept, lazy or time-poor [7], [36], [37].”

• cites [7] 7. M. Joy and M. Luck, ''Plagiarism in programming assignments,'' IEEE Trans. Educ., vol. 42, no. 2, pp. 129-133, May 1999. sourceNot checked — no full text of the source
• cites [36] 36. J. Sheard, S. Markham, and M. Dick, ''Investigating differences in cheat- ing behaviours of IT undergraduate and graduate students: The maturity and motivation factors,'' Higher Educ. Res. Develop., vol. 22, no. 1, pp. 91-108, May 2003. sourceNot checked — no full text of the source
• cites [37] 37. Simon, T. Myers, D. Hardy, and R. Mason, ''Variations on a theme: Academic integrity and program code,'' in Proc. 21st Australas. Comput. Educ. Conf. (ACE). New York, NY, USA: Association for Computing Machinery, 2019, pp. 56-63, doi: 10.1145/3286960.3286967. sourceNot checked — no full text of the source

The three analysed aspects are sufficient for representing behaviour for source code plagiarism detection.

“However, these three analysed aspects are sufficient for representing behaviour for source code plagiarism detection.This will be demonstrated through the evaluation of the proposed SCPDT, BPlag, that implements this model (Section V).”

Analysing program behaviour is expected to be most beneficial for detecting plagiarism involving pervasively applied structural transformations.

“Analysing program behaviour is expected to be most beneficial in the detection of plagiarism with pervasively applied structural transformations.”

Using concrete execution to analyse behaviour would require many test inputs to record multiple execution paths, which was considered infeasible for a source code plagiarism detection tool.

“If concrete execution (i.e.'conventional' execution) was applied to analyse behaviour, it would require providing a large set of test inputs to record multiple execution paths under different input sets.This was deemed infeasible for a SCPDT.”

It is much harder to transform a program's behaviour to evade detection than to transform its structure.

“While it is easy to transform the structure of a program, it is argued that it is much more difficult to transform the behaviour of a program to evade detection.”

Achieving such behavioural change would effectively require rewriting significant portions of the misappropriated program.

“To do so, the plagiariser would effectively have to rewrite significant portions of the mis-appropriated program.”

Symbolic execution makes it easy to record all potential execution paths.

“However, recording all potential execution paths is easily afforded with symbolic execution.”

All submissions for the same assessment task are inherently behaviourally similar.

“All assignment submissions implementing the same assessment task are all inherently behaviourally similar.This is as they implement the same assignment specification, meaning it can be expected that all submissions for the same assessment task will accept the same inputs, and if correctly implemented, produce the same outputs.”

BPlag is theoretically robust to L6 transformations provided they do not change the behavioural representation of a program.

“BPlag is in theory, robust to L6 transformations on the condition that such transformations do not change the behavioural representation of a program.”

A custom-built tool that omits constraint solving and library bytecode execution was judged to be more efficient.

“Hence, a custom built tool that omits constraint solving and library bytecode execution was deemed to provide greater efficiency.”

It is unlikely that a plagiariser would apply functionally equivalent but behaviourally distinct transformations through an entire program because it is time-intensive and requires high skill.

“This is considered unlikely as rewriting a program to be functionally equivalent but behaviourally distinct would be a time-intensive task for a skilled but lazy plagiariser, and would require a level of programming skill above that expected from an inept plagiariser.”

It is expected that functionally equivalent but behaviourally distinct transformations would need to be applied through an entire program to evade BPlag.

“However, it is expected that such transformations would have to be applied through an entire program in order to evade detection by BPlag.”

Functionally equivalent but behaviourally distinct code can evade BPlag.

“Hence, there is potential for such transformations to evade BPlag by introducing functionally equivalent code; that subsequently has a distinct behavioural representation.”

Changing decision logic in a program does not grossly affect BPlag as long as the same code is executed.

“Changing the decision logic of a program would not grossly affect BPlag, as long as it results in the same code executed.”

If L6 transformations introduce functionally-equivalent but behaviourally-distinct code, BPlag's similarity evaluation is expected to be impacted.

“However, if transformations introduce functionally-equivalent but behaviourally-distinct code, BPlag is expected to see an impact on the evaluation of similarity.”

Modifying expressions can have a large impact on BPlag's similarity evaluation.

“Conversely, modifying expressions can have a large impact on the approach.”

Existing symbolic execution tools execute all Java bytecode, including system library bytecode.

“Furthermore, existing symbolic execution tools execute all Java bytecode, even that provided as part of system libraries.”

Flipping a relational operator in an if condition results in a minor difference in behaviour for BPlag.

“For example, if a relational operator is flipped as part of an if condition, it would result in a minor difference in behaviour.”

The overlap coefficient does not consider whether filtered graph nodes are legitimately part of a program or injected noise.

“The overlap coefficient does not consider if the filtered graph nodes are legitimately part of a program, or simple injected noise.”

The absence of a robustness decrease for L4 and L5 transformations implies BPlag is robust to those transformations.

“The lack of decrease in robustness to L4 and L5 transformations implies that BPlag is robust to such transformations.”

• AssumptionRests on an assumption: L4 and L5 transformations do not modify the behaviour of a program.

This noise results in a decrease in similarity.

“Hence, this results in a decrease in similarity.”

The overlap coefficient can be used to effectively ignore the addition of new code.

“Similarity, the overlap coefficient could be used to effectively ignore the addition of new code.”

This introduces new data nodes to the graphs that cannot be matched to the base program.

“As a result of this, it introduces new data nodes to the graphs that cannot be matched to the base program.”

If all similarity scores are plotted on a number line, errors make it impossible to linearly separate plagiarised and innocent pairs into homogeneous groups.

“Subsequently, if all similarity scores for a data set are plotted on a number line, this results in it being impossible to linearly separate those that are plagiarised and innocent into homogeneous groups.”

If two programs have high similarity, it can be considered an indication that plagiarism may be present.

“If two programs have a high similarity, it can be considered an indication that plagiarism may be present.”

Lossy filtering and overlap coefficient allow BPlag to largely accommodate value-injecting transformations.

“Through the introduction of lossy filtering, and similarity evaluation through the overlap coefficient, it was shown that BPlag can largely accommodate for value-injecting transformations.”

This vulnerability was largely mitigated through noise filtering accommodation.

“However, this was shown to be largely mitigated through the noise filtering accommodation.”

Because the applied transformations are mostly structural and cosmetic, they have a considerable impact on the tools' ability to measure similarity.

“As the applied source code transformations are mostly structural and cosmetic in nature, they are shown to have a considerable impact on the tools ability to measure similarity.”

• AssumptionRests on an assumption: The applied source code transformations are mostly structural and cosmetic in nature.

BPlag is much more robust to transformation than the SCPDTs.

“Hence, it can be concluded that BPlag is much more robust to transformation compared to the SCPDTs; however, it does have a vulnerability to value-injecting transformations, that subsequently introduce noise into the PIDGs.”

BPlag is more robust to plagiarism-hiding transformations.

“In summary of these results, BPlag is shown to be more robust to plagiarism-hiding transformations.”

Value injection is a vulnerability of all SCPDTs, not only BPlag.

“It should be noted that value injection is a vulnerability of all SCPDTs, and not just BPlag.”

Different interpretations of high similarity have led to different approaches for evaluating SCPDTs.

“Hence, this has resulted in different interpretations of what is considered an indication of plagiarism for use in SCPDT evaluations, and how to evaluate SCPDTs in general [9].”

• cites [9] 9. M. Novak, M. Joy, and D. Kermek, ''Source-code similarity detection and detection tools used in academia: A systematic review,'' ACM Trans. Comput. Educ., vol. 19, no. 3, pp. 1-37, Jun. 2019. sourceNot checked — no full text of the source

From Fig. 5 results, BPlag demonstrates greater robustness to plagiarism-hiding transformations.

“From the results in Fig. 5, BPlag demonstrates greater robustness to plagiarism-hiding transformations.”

Value-injecting transformations add noise that manifests as increased node and edge counts in PIDGs.

“Logically, value-injecting transformations add 'noise', that manifests as increased node and edge counts of the PIDGs representing a program.”

This indicates errors in the graph-matching algorithm caused by lossy added noise.

“This would indicate the occurrence of errors in the graph-matching algorithm, as a result of the 'lossy' of added noise.”

BPlag has a vulnerability to value-injecting transformations that introduce noise into PIDGs.

“Hence, it can be concluded that BPlag is much more robust to transformation compared to the SCPDTs; however, it does have a vulnerability to value-injecting transformations, that subsequently introduce noise into the PIDGs.”

This allows accurate evaluation of SCPDT accuracy with a ground-truth data set with known plagiarism-hiding transformations.

“Hence, this allows the accurate evaluation of SCPDT accuracy with a ground-truth data set with known types of plagiarismhiding transformations applied.”

These tools measure similarity through source code aspects that are modified by undergraduate plagiarisers.

“This is an important observation as these tools measure similairty through aspects of source code that are modified by undergraduate plagiarisers.”

• AssumptionRests on an assumption: The applied source code transformations are representative of those used by undergraduate plagiarisers.

The five compared SCPDTs are less robust to the applied source code transformations because they measure structural similarity of source code.

“The five compared SCPDTs are less robust to the applied source code transformations as they measure the structural similarity of source code.”

BPlag's vulnerability to L3 transformations results from value-injecting transformations that add new values to PIDGs.

“On further analysis, this is a result of 'value-injecting transformations': transformations that result in new values represented on PIDGs.”

Using generated test data allows construction of a data set with known plagiarised pairs that can be injected into innocent assignment submissions.

“Using generated test data allows for the construction of a data set with known plagiarised programs pairs that can be injected into an existing data set of innocent assignment submissions.”

Measuring behavioural similarity yields greater robustness against source code transformations representative of undergraduate plagiarisers.

“Overall, these results have shown that by measuring the behavioural similarity of the programs, greater robustness can be gained against source code transformations that are representative of undergraduate plagiarisers.”

• AssumptionRests on an assumption: The applied source code transformations are representative of those used by undergraduate plagiarisers.

If two PIDGs are vastly different in size but one is a subset of the other, the overlap coefficient can produce a false positive through an unexpectedly high similarity score.

“Furthermore, if two PIDGs are of vastly different sizes, but one is a subset of the other, it can result in a false positive through the evaluation of an unexpectedly high similarity score.”

Some errors are expected because lossy filtering is applied to a graph-matching heuristic.

“However, as this is a lossy filtering method applied to a graph-matching heuristic, the introduction of some errors is expected.”

Analysing functional equivalence is not suitable for SCPD because it does not consider program design and implementation.

“However, as discussed, analysing functional equivalence is not suitable for SCPD as it does not consider the design and implementation of a program.”

With lossy filtering and overlap coefficient, BPlag can gain even greater robustness to plagiarism-hiding transformations.

“However, overall, these results show that in combination with the lossy filtering method and overlap coefficient, BPlag can gain even greater robustness to plagiarism-hiding transformations.”

When a conventional SCPDT evaluates similarity with injected source code, it cannot match the injected code and similarity drops.

“When a conventional SCPDT attempts to evaluate similarity in such a case, it cannot match the injected source code and hence results in a drop in similarity.”

Value-injecting transformations are the most apparent weakness of BPlag.

“Value-injecting transformations are the most apparent weakness of BPlag.”

This results in a decrease in similarity.

“Hence, it results in a decrease in similarity.”

Experiment 1 demonstrated that BPlag is largely robust to 20% and 40% transformation chances.

“Higher transformation chances are used in this experiment, as experiment 1 demonstrated that BPlag is largely robust to the 20% and 40% transformation chances.”

Greater accuracy can be gained by measuring similarity through source code behaviour when detecting source code plagiarism hidden with plagiarism-hiding transformations.

“Overall, this demonstrates that greater accuracy can be gained by measuring similarity through the behaviour of source code, when detecting source code plagiarism hidden with plagiarism-hiding transformations.”

Dependence of results on the composition of the base programs affects every evaluation that measures source code similarity.

“However, this problem is common to all evaluations that measure source code similarity”

The generated plagiarised variants can be regarded as representative of the work produced by undergraduate plagiarisers.

“Hence, the variants can be said to be representative of undergraduate plagiarisers.”

• AssumptionRests on an assumption: The way SPPlagiarise applies its transformations matches how undergraduate plagiarisers apply transformations.

Analysing behaviour renders structural plagiarism-hiding modifications largely inert.

“As demonstrated in experiment 1, by analysing behaviour, the plagiarism-hiding modifications that transform the structure of source code are rendered largely inert.”

The value-injecting vulnerability was accommodated using noise filtering and overlap coefficient.

“However, in the latter case, it was shown to be accommodated for using a simple filtering of noise from the generated PIDGs, and the use of the overlap coefficient to evaluate similarity.”

This decrease is explained by the value-injecting transformations identified in experiment 1.

“In theory BPlag should not be affected by such transformations, however, this is again explained by the value-injecting transformations identified in experiment 1.”

Plagiarism-hiding transformations do not significantly impact the behavioural representation of a program.

“This is as the plagiarism-hiding transformations do not have a significant impact on the behavioural representation of a program.”

BPlag is vulnerable to transformations that alter a program's behavioural representation, namely the introduction of functionally equivalent code and value-injecting transformations.

“As identified, BPlag is vulnerable to transformations that change the behavioural representation of a program.”

• Also stated in: § B. VULNERABILITY TO TRANSFORMATIONS · plain

  “But, BPlag is vulnerable to the identified introduction of functionally-equivalent code, or value-injecting transformations.”

As a trade-off for greater robustness and accuracy, BPlag is less efficient than the five compared SCPDTs due to greater complexity.

“As a trade-off for greater robustness and accuracy, BPlag is less efficient than the five compared SCPDTs due to the greater complexity of the approach.”

The greater runtime does not make BPlag impractical to use.

“The greater runtime does not mean BPlag is impractical to use.”

BPlag's poorer efficiency is attributed to its comparatively more complex implementation.

“The poorer efficiency is attributed to the comparatively more complex implementation of the approach.”

Using generated data sets allows a known number of plagiarised programs to be generated with well-known types of plagiarism-hiding transformations.

“By using generated data sets, a known number of plagiarised programs can be generated with well-known types of plagiarism-hiding transformations.”

BPlag's vulnerability to functionally equivalent code is a necessary weakness, accepted in order to prevent false positive plagiarism detections.

“However, the vulnerability to functionally equivalent code is a required deficiency of BPlag to avoid false positive errors when detecting plagiarism.”

BPlag is more accurate in detecting plagiarism where the plagiarised work is pervasively transformed with plagiarism-hiding transformations.

“Subsequently, the approach is more accurate in the detection of plagiarism, where the plagiarised work is pervasively transformed with plagiarismhiding transformations, as demonstrated in experiment 2.”

Greater accuracy is also influenced by the approach representing the unique implemented behaviour of a program as expressed through source code.

“However the greater accuracy is also influenced by the approach representing the unique implemented behaviour of a program, as expressed through source code.”

BPlag is expected to withstand the source code transformations that are typical of undergraduate plagiarisers.

“Hence, it should be expected that BPlag is robust against source code transformations characteristic of undergraduate plagiarisers.”

• AssumptionThe authors state this as an assumption: BPlag can cope with plagiarism-hiding transformations that are cosmetic or structural, and with those that preserve semantics and behaviour.

By analysing program behaviour, BPlag consistently measures higher similarity between simulated plagiarised programs than the five compared SCPDTs.

“By analysing program behaviour, it is capable of consistently measuring a higher similarity between the simulated plagiarised programs, as compared to the five compared SCPDTs.”

SPPlagiarise provides 19 source code transformations, which correspond to levels L1 through L5 of the taxonomy of Faidhi and Robinson.

“SPPlagiarise implements a total of 19 source code transformations conforming to L1-L5 of Faidhi and Robinson's [10] taxonomy.”

• cites [10] 10. J. A. W. Faidhi and S. K. Robinson, ''An empirical approach for detecting program similarity and plagiarism within a university programming envi- ronment,'' Comput. Educ., vol. 11, no. 1, pp. 11-19, Jan. 1987. sourceNot checked — no full text of the source

Unavoidable errors occur when identifying the plagiarised programs.

“Hence, there are unavoidable errors that occur when identifying the plagiarised programs.”

In this experiment the data-set dependence problem is reduced by using a large and diverse data set.

“and is mitigated in this experiment with a large and diverse data set.”

Undergraduate plagiarisers are not expected to carry out either attack, since it demands substantial programming skill and/or time.

“Neither attacks are expected from undergraduate plagiarisers as it would require considerable programming skills and/or time.”

Greater accuracy largely results from greater robustness, allowing BPlag to evaluate higher similarity between plagiarised works with pervasive transformations than innocent works.

“Greater accuracy is largely a result of the greater robustness, allowing for BPlag to evaluate a higher similarity between plagiarised works (with pervasive plagiarism-hiding transformations) than innocent works.”

This makes the similarity scores of plagiarised and innocent program pairs no longer linearly separable.

“This effectively results in the similarity scores of the plagiarised and innocent program pairs becoming no longer linearly separable.”

Implementing the three stages as separate programs allows the approach to run without garbage collection overhead.

“By implementing the three stages as separate programs, it effectively allows the approach to run without any garbage collection overhead.”

Errors are caused by some variant similarity scores dropping to a point where correctly innocent scores are ranked higher.

“This is caused by the similarity scores of some variant programs dropping to a point where correctly innocent scores are ranked higher.”

With accommodations for noise, BPlag achieves greater accuracy against value-injecting transformations.

“Hence, by integrating the accommodations for the introduction of noise, BPlag is shown to achieve greater accuracy against the value-injecting transformations.”

The reported results establish BPlag's performance conclusively only for the data set that was used.

“Therefore, the results presented here can only conclusively determine the performance of BPlag on the utilised data set.”

BPlag identifies similar programs by examining runtime behaviour, specifically data and its relations, transformations of data, and method calls.

“While BPlag seeks to identify similar programs by analysing how the program behaves at runtime by identifying data and its relations, transformation of data, and method calls.”

As a general rule, every source code plagiarism detection tool is vulnerable to transformations that change the program representation that tool relies on.

“However, as a generalisation, it must be considered that all SCPDTs are vulnerable to transformations that change the representation of a program used by the SCPDT.”

The transformations typically used to hide plagiarism have little effect on BPlag, apart from those that change the behavioural representation of the programs.

“Hence, the characteristic plagiarism-hiding transformations do not have a significant impact on the approach, except for those that modify the behavioural representation of the programs.”

BPlag's approach has nothing in common with existing approaches to source code plagiarism detection.

“The approach does not share similarity with existing approaches to SCPD.”

Any modification of the program representation a similarity tool uses necessarily lowers the similarity that tool measures.

“When the representation of a program is modified, it is inevitable any source code similarity tool will measure a decrease in similarity.”

This could become a source of errors if BPlag were applied to low-level languages.

“This has potential to be a source of errors if the approach were to be re-applied to low-level languages.”

The main difference between BPlag and these tools is the form in which the program is analysed: BPlag works from source code to build a behavioural representation, whereas the compared tools work from machine code or bytecode.

“The major difference between BPlag and these tools is the form a program takes when analysed.BPlag analyses source code to derive a behavioural representation, while the compared works analyse machine code or bytecode.”

BPlag is a clearly behavioural approach to academic source code plagiarism detection.

“BPlag is a distinctly behavioural approach to academic SCPD.”