The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; none of the 34 sources its claims cite is checked for support yet.
Battle of the Attack Detection Algorithms: Disclosing Cyber Attacks on Water Distribution Networks
Open accessAbstract
The BATtle of the Attack Detection ALgorithms (BATADAL) is the most recent competition on planning and management of water networks undertaken within the Water Distribution Systems Analysis Symposium. The goal of the battle was to compare the performance of algorithms for the detection of cyber-physical attacks, whose frequency has increased in the last few years along with the adoption of smart water technologies. The design challenge was set for the C-Town network, a real-world, medium-sized water distribution system operated through programmable logic controllers and a supervisory control and data acquisition (SCADA) system. Participants were provided with data sets containing (simulated) SCADA observations, and challenged to design an attack detection algorithm. The effectiveness of all submitted algorithms was evaluated in terms of time-to-detection and classification accuracy. Seven teams participated in the battle and proposed a variety of successful approaches leveraging data analysis, model-based detection mechanisms, and rule checking. Results were presented at the Water Distribution Systems Analysis Symposium (World Environmental and Water Resources Congress) in Sacramento, California on May 21–25, 2017. This paper summarizes the BATADAL problem, proposed algorithms, results, and future research directions.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
BATADAL was a competition comparing seven attack-detection algorithms on simulated SCADA data from the C-Town water distribution network under cyber-physical attacks. The winning model-based approach by Housh and Ohar achieved the best overall ranking score (S = 0.970), while most algorithms outperformed naïve detection baselines.
Key findings
- Seven teams participated; all but one achieved a ranking score S > 0.75, better than naïve detection mechanisms.
- Housh and Ohar ranked first with S = 0.970, S_TTD = 0.965, S_CLF = 0.975, detecting all 7 test attacks; Abokifa et al. ranked second with S = 0.949.
- The only model-based approach won: estimating water demands, simulating system dynamics with EPANET, and measuring errors against SCADA readings proved successful.
- Aghashahi et al. ranked last with S = 0.534 and detected only 3 attacks; Chandy et al. was over-sensitive with many false alarms, while Pasha et al. lacked sensitivity.
Methods
- Design challenge used the C-Town network: 429 pipes, 388 junctions, 7 storage tanks, 11 pumps, 5 valves, 1 reservoir, and 9 PLCs coordinated by SCADA.
- Participants received three simulated SCADA datasets: Training dataset 1 with no attacks over 365 days at hourly time steps; Training dataset 2 with seven attacks over 492 hourly time steps; and a Test dataset with seven attacks over 407 hourly time steps.
- Algorithms were evaluated by time-to-detection (S_TTD), classification performance (S_CLF = mean of TPR and TNR), and overall ranking score S = 0.5·S_TTD + 0.5·S_CLF.
Limitations
- The paper states the evaluation was deterministic on three specific datasets representing only a small portion of possible cyber-attacks; robustness would require stochastic scenarios with varying hydraulic conditions and multiple attack sequences.
- The paper states a major limitation is the absence of detailed real SCADA data on cyber attacks to water utilities, and that BATADAL relied on data generated with a demand-driven engine.
- Evident from the text: participants were given the same computational model of C-Town used to generate the SCADA data, and demand patterns were fairly regular and consistent across datasets, which may limit real-world applicability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
22 No full text of the sources · 14 Own interpretation · 8 No basis given · 5 Sources not found · 4 Own method · 3 Own result · 2 Not checked yet
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Interestingly, the BATADAL was won by the only model-based approach.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“With a value of S equal to 0.896, the algorithm proposed by Brentan et al. (2017) can also be regarded as a strong performer.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Brentan et al. (2017) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In this regard, it is important to note that BATADAL demand patterns were fairly regular and consistent across the three datasets.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that a naïve detection mechanism that predicts the system to be always in safe conditions gets a score S equal to 0.25 (S T T D = 0, S CLF = 0.5).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Ranking score
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The algorithm ofAbokifa et al. (2017) comes a close second, with S equal to 0.949.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > SUPPLEMENTAL DATA
What the sources say
- Not checked — no full text of the sourcecites Abokifa et al. (2017) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“With the exception of Brentan et al. (2017) and Pasha et al. (2017), there appears to be a strong correlation between these two metrics (see Fig -ure 3).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
What the sources say
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The group is led by the algorithm proposed byHoush and Ohar (2017b), which shows the best overall performance (S = 0.970).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > SUPPLEMENTAL DATA
What the sources say
- Not matchedcites Housh and Ohar (2017b)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“As shown in Figure 4(b), Abokifa et al. (2017) algorithm disclosed Attack #10 and #11 as a single continuous episode, erroneously flagging the system as under attack for the period in between.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Abokifa et al. (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Only a few algorithms provided information on the attacked devices.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Among these, the algorithms proposed by Brentan et al. (2017) and Giacomoni et al. (2017) were the most accurate.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 20
- Probably not a substantial claim. 20
In the paper
“As shown in Figure 4(d), this algorithm was able to consistently and accurately detect most of the attacks, but it failed to identify the last one.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Brentan et al. (2017) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“However, Giacomoni et al. (2017) algorithm is less sensitive, resulting in a higher number of false negatives and minor timing errors (see Figure 4(c)) that lead to a score S equal to 0.927.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Giacomoni et al. (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Their score S is equal to 0.802 and 0.773, respectively.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“On the other hand, the algorithm of Pasha et al. (2017) issued just a few false alarms, but it lacked sensitivity, thus failing to flag the system as under attack for the entire duration of the events.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Pasha et al. (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Indeed, the detection trajectory depicted in Figure4(a) shows that all attacks were immediately detected, with the exception of the last one, which was disclosed a few hours after its starting time.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > SUPPLEMENTAL DATA
What the sources say
- Not matchedcites Housh and Ohar (2017b)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“The algorithm proposed by Giacomoni et al. (2017) has the same number of false positives and true negatives as that of Housh and Ohar (2017b)-meaning that both algorithms were the most successful in avoiding false alarms.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Giacomoni et al. (2017) source
- Not matchedcites Housh and Ohar (2017b)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Note that this algorithm is the top scorer in terms of both time-to-detection S T T D and classification score S CLF .”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > SUPPLEMENTAL DATA
What the sources say
- Not matchedcites Housh and Ohar (2017b)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“since the number of attacks in the available data is generally limited.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“The algorithm of Chandy et al. (2017) turned out to be over-sensitive-meaning that it was able to identify most of the attack instances, but at the cost of issuing numerous false alarms.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Chandy et al. (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Seven teams participated in the battle and proposed a variety of successful approaches leveraging data analysis, model-based detection mechanisms, and rule checking.”
- Also stated in: § ATTACK DETECTION ALGORITHMS · plain
“Seven teams participated in BATADAL.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Also stated in: § ATTACK DETECTION ALGORITHMS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All algorithms but one achieved a ranking score S larger than 0.75, meaning that they performed better than naïve detection mechanisms.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On the other hand, flagging the system as always under attack yields a value of S equal to 0.75 (S T T D = 1, S CLF = 0.5).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Ranking score
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“The definition of the cut-off criteria defining outliers regulates the trade-off between True Positive and True Negative Rate for most of the algorithms”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > FUTURE RESEARCH DIRECTIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Finally, the contribution of Aghashahi et al. (2017) detected only three attacks, leading to a score S equal to 0.534.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
What the sources say
- Not checked — no full text of the sourcecites Aghashahi et al. (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Most teams presented multi-stage detection methods.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Protecting water distribution systems from cyber attacks requires (as with other cyberphysical systems) a combination of proactive and reactive mechanisms (Cardenas et al. 2008).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites Cardenas et al. 2008 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“according to which the number of reported attacks on water infrastructures has been growing steadily (ICS-CERT 2014; ICS-CERT 2015; ICS-CERT 2016)-making them the third highest targeted sector after critical manufacturing and energy (ICS-CERT 2016).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > INTRODUCTION
What the sources say
- Not matchedcites ICS-CERT 2014
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites ICS-CERT 2015
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites ICS-CERT 2016
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“The adoption of such smart water technologies plays a pivotal role in enhancing the reliability and autonomy of water distribution systems, but simultaneously exposes them to cyber-physical attacks (Rasekh et al. 2016)”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites Rasekh et al. 2016 source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This means that the information provided by SCADA systems may not be fully reliable.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > INTRODUCTION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Rests on an assumption: All networked devices, including SCADA systems, represent potential targets.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Comparing and confirming the detection issued by different modules can help decrease classification errors.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can probably conclude that both model-based and data-driven approaches are suitable for attack detection problems, although their performance would probably vary with the modelling context at hand.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“While the data and algorithms presented here provide a first step towards an objective comparison of attack detection algorithms for water distribution systems, they do not represent the entire spectrum of modelling contexts that practitioners and researchers would encounter.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > CLOSURE
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since attack detection mechanisms aim to disclose outliers and contextual anomalies in the system behavior, they may accidentally disclose anomalous behaviors that are not necessarily caused by cyber attacks.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > FUTURE RESEARCH DIRECTIONS
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The latter were aimed at manipulating the information sent or received by sensors and PLCs, with the ultimate goal of affecting the operations of an actuator (Urbina et al. 2016).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Development data
What the sources say
- Not checked — no full text of the sourcecites Urbina et al. 2016 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The evaluation of BATADAL algorithms can be seen as a deterministic analysis carried out on three specific datasets, which represent only a small portion of the entire set of cyber-attacks that could threaten a water distribution system.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > FUTURE RESEARCH DIRECTIONS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The first two datasets, hereafter named Training dataset 1 and Training dataset 2, were provided at the beginning of the competition, while the third one (Test dataset) was subsequently used to evaluate and rank the attack detection algorithms.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Development data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both time-to-detection and classification score are important aspects of performance.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is a rather challenging task due to the intrinsic correlation among the hydraulic variables.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > FUTURE RESEARCH DIRECTIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The solutions provided by seven teams suggest that timely and accurate detection can be obtained by both modelbased and data-driven approaches, usually made of multiple sequential stages.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > CLOSURE
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“SCADA concealment was performed by either replacing actual traffic information between PLCs and SCADA with previously-recorded data (replay attacks) or adding an offset to the transmitted sensor readings (Urbina et al. 2016).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Development data
What the sources say
- Not checked — no full text of the sourcecites Urbina et al. 2016 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“A limitation of this battle is its reliance of data generated with a demand-driven engine (Taormina et al. 2017).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > FUTURE RESEARCH DIRECTIONS
What the sources say
- Not checked — no full text of the sourcecites Taormina et al. 2017 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It appears that consistency checks and the analysis of control rules should lead to the identification of the simplest attacks.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“A key aspect of the dataset is the absence of cyber attacks, which made it suitable for studying the operations of the water distribution system under normal operating conditions.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Development data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The operations of the water system were altered through malicious activation of hydraulic actuators, change of actuator settings, and deception attacks-amongst the most common for cyber-physical systems (Cardenas et al. 2009).”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Development data
What the sources say
- Not matchedcites Cardenas et al. 2009
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The inherent interdependence of the elements in the water network should theoretically allow for the detection of anomalies, even when the adversary tries to conceal his (her) actions by altering the SCADA readings of one or a few deployed sensors.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > General Observations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
All networked devices, including SCADA systems, represent potential targets.
“Third, all networked devices, including SCADA systems, represent potential targets.”
The paper relies on this without support.
It is not possible to rule out all attacks.
“Since it is not possible to rule out all attacks, cyber-physical systems should also be equipped with intrusion detection schemes that assist with the recovery phase (Anderson 2010).”
The paper relies on this without support.
The simulated SCADA data and attacks in the BATADAL datasets adequately represent real-world cyber-physical attacks on water distribution networks.
“Participants were provided with datasets containing (simulated) SCADA observations, and challenged with the design of an attack detection algorithm.”
“The datasets were generated via simulation with epanetCPA, a Matlab toolbox that allows to design a variety of cyber attacks and simulate, with EPANET2 (version 2.0.12), the hydraulic response of a water distribution network; see Taormina et al. (2017) for further details.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > Algorithms performance
“With a value of S equal to 0.896, the algorithm proposed by Brentan et al. (2017) can also be regarded as a strong performer.”
§ THE BATTLE OF THE ATTACK DETECTION ALGORITHMS 1 > SUPPLEMENTAL DATA
“| 4 | Brentan et al. | 6 | 0.894 0.857 0.931 362 45 1637 45 | | |”
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515
Tutorial on Variational Autoencoders
Doersch, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05908 · Cited by 1,470
Diagnosing network-wide traffic anomalies
Lakhina, A., Crovella, M., & Diot, C. (2004) · Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication · doi:10.1145/1015467.1015492 · Cited by 1,399
Secure Control: Towards Survivable Cyber-Physical Systems
Cardenas, A. A., Amin, S., & Sastry, S. (2008) · 2008 The 28th International Conference on Distributed Computing Systems Workshops · doi:10.1109/icdcs.workshops.2008.40 · Cited by 911
Event detection from time series data
Guralnik, V., & Srivastava, J. (1999) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/312129.312190 · Cited by 480
Security Engineering: A Guide to Building Dependable Distributed Systems
Anderson, R. (2020) · doi:10.1002/9781119644682 · Cited by 345
Anomaly Detection via Online Oversampling Principal Component Analysis
Lee, Y., Yeh, Y.-R., & Wang, Y.-C. F. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.99 · Cited by 225
An evaluation framework for input variable selection algorithms for environmental data-driven models
Galelli, S., Humphrey, G. B., Maier, H. R., Castelletti, A. F., Dandy, G. C., & Gibbs, M. S. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.08.015 · Cited by 208
Self-citation
Research works printed but not linked: 2
- Cardenas A. S. Amin B. Sinopoli A. Giani A. Perrig and S. Sastry. 2009. “Challenges for securing cyber physical systems.” In Vol. 5 of Proc. Workshop on Future Directions in Cyber-Physical Systems Security. Washington DC: Department of Homeland Security.
- Rossman L. A., 2000, EPANET 2 users manual
Cited by
Sorted by citations
Tyralis, H., Papacharalampous, G., & Langousis, A. (2019) · Water · doi:10.3390/w11050910 · Cited by 828
The role of deep learning in urban water management: A critical review
Fu, G., Jin, Y., Sun, S., Yuan, Z., & Butler, D. (2022) · Water Research · doi:10.1016/j.watres.2022.118973 · Cited by 479
“…14 attack scenarios considering various components such as pumps, tanks and controllers in a benchmark water distribution system (i.e., C-Town), and results showed autoencoders substantially outperform traditional machine learning methods, i.e., XGBoost and LightGBM (Taormina and Galelli, 2018).”
BackgroundMethodInfluentialAn Ensemble Deep Learning-Based Cyber-Attack Detection in Industrial Control System
Al-Abassi, A., Karimipour, H., Dehghantanha, A., & Parizi, R. M. (2020) · IEEE Access · doi:10.1109/access.2020.2992249 · Cited by 323
“While the security concerns of critical infrastructure facilities are already considered in the IT community, limited efforts have been made to develop security solutions that are specific to ICSs and OT environments [9].”
BackgroundA Survey on Industrial Control System Testbeds and Datasets for Security Research
Conti, M., Donadel, D., & Turrin, F. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3094360 · Cited by 284
Kravchik, M., & Shabtai, A. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3050101 · Cited by 224
“Figure 7 shows that the status of the valve was not replayed, although the authors of [37] reported that it was.”
BackgroundMethodInfluential
Metadata
- Authors
- Taormina, R., Galelli, S., Tippenhauer, N. O., Salomons, E., Ostfeld, A., Eliades, D. G., Aghashahi, M., Sundararajan, R., Pourahmadi, M., Banks, M. K., Brentan, B. M., Campbell, E., Lima, G., Manzi, D., Ayala-Cabrera, D., Herrera, M., Montalvo, I., Izquierdo, J., Luvizotto, E., Chandy, S. E., Rasekh, A., Barker, Z. A., Campbell, B., Shafiee, M. E., Giacomoni, M., Gatsis, N., Taha, A., Abokifa, A. A., Haddad, K., Lo, C. S., Biswas, P., Pasha, M. F. K., Kc, B., Somasundaram, S. L., Housh, M., & Ohar, Z.
- Affiliations
- Singapore University of Technology and Design (SG)
- Independent Water Resources Consultant, OptiWater, 6 Amikam Israel St., Haifa 3438561, Israel.
- Technion – Israel Institute of Technology (IL)
- University of Cyprus (CY)
- Texas A&M University (US)
- CRAN (FR)
- Berliner Wasserbetriebe (Germany) (DE)
- Universidade Estadual de Campinas (UNICAMP) (BR)
- Research Fellow, Dept. of Water, Irstea, BP 5095, 34196 Cestas, France.
- University of Bath (GB)
- Research Engineer, Ingeniousware GmbH, 76137 Karlsruhe, Germany.
- Universitat Politècnica de València (ES)
- Data Scientist, Sensus, Inc., 8601 Six Forks Rd., Suite 700, Raleigh, NC 27615.
- Lead R&D Engineer, Sensus, Inc., 8601 Six Forks Rd., Suite 700, Raleigh, NC 27615.
- Water Resources Engineer, Sensus, Inc., 8601 Six Forks Rd., Suite 700, Raleigh, NC 27615.
- Chief Data Scientist, Sensus, Inc., 8601 Six Forks Rd., Suite 700, Raleigh, NC 27615.
- The University of Texas at San Antonio (US)
- Washington University in St. Louis (US)
- California State University, Fresno (US)
- University of Haifa (IL)
- Corresponding author
- Galelli, Stefano
- ORCID
- Taormina, Riccardo
- Galelli, Stefano
- Tippenhauer, Nils Ole
- Salomons, Elad
- Ostfeld, Avi
- Eliades, Demetrios G.
- Sundararajan, Raanju
- Brentan, B. M.
- Campbell, Enrique
- Herrera, M.
- Montalvo, I.
- Izquierdo, J.
- Luvizotto, E.
- Rasekh, Amin
- Campbell, Bruce
- Shafiee, M. Ehsan
- Giacomoni, Marcio
- Gatsis, Nikolaos
- Taha, Ahmad
- Abokifa, Ahmed A.
- Lo, Cynthia S.
- Biswas, Pratim
- Pasha, M. Fayzul K.
- Kc, Bijay
- Housh, Mashor
- Venue
- Journal of Water Resources Planning and Management
- Published
- 2018-08
- Volume
- 144
- Issue
- 8
- ISSN
- 0733-9496 (ISSN-L)
- 1943-5452
- OpenAlex
- W2807219356
- Semantic Scholar
- 64815d88bf9bfb49e6e9e2bb9be6eac3d56ea7f6
- Cited by
- 277
- References
- 33
- Citations by year
- 2026: 49 · 2025: 45 · 2024: 38 · 2023: 31 · 2022: 29 · 2021: 36 · 2020: 25 · 2019: 19 · 2018: 5
- FWCI
- 13.14
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Water Resources Planning and Management (publishedVersion)
- Pure (University of Bath) (submittedVersion, other-oa)
- RiuNet (Universitat Politècnica de València) (submittedVersion, cc-by-nc-nd)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- Greater South Information System
- Funders
- Washington University in St. Louis
- European Commission (739551, HORIZON2020)
- National Research Foundation (NRF2014NCR-NCR001-40)
- Fonds National de la Recherche Luxembourg
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Qatar National Research Fund (NPRP8-1292-2-548)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Smart Grid Security and Resilience
- Physical Sciences › Engineering › Civil and Structural Engineering › Water Systems and Optimization
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- cyberattack detection, water distribution systems, cyber-physical attacks, SCADA, programmable logic controller, attack detection algorithm, algorithm comparison, detection time, classification accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-21, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-21 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
