Stackelberg games for adversarial prediction problems
The DOI matches its Crossref record; 18 of the 19 references it cites were found in a registry or an index.
Abstract
The standard assumption of identically distributed training and test data is violated when test data are generated in response to a predictive model. This becomes apparent, for example, in the context of email spam filtering, where an email service provider employs a spam filter and the spam sender can take this filter into account when generating new emails. We model the interaction between learner and data generator as a Stackelberg competition in which the learner plays the role of the leader and the data generator may react on the leader's move. We derive an optimization problem to determine the solution of this game and present several instances of the Stackelberg prediction game. We show that the Stackelberg prediction game generalizes existing prediction models. Finally, we explore properties of the discussed models empirically in the context of email spam filtering.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Evasion attacks can be protected against by devising systems composed of multiple classifiers, as outlined in [137], or other adversary-aware learning algorithms (e.g.one-and-a-halfclass multiple classifiers) as outlined in [135].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [137]
Citations
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Wächter, A., & Biegler, L. T. (2006) · Mathematical Programming · doi:10.1007/s10107-004-0559-y · Cited by 9,483 · cited 1×
“To solve the nonlinear program of the SPG with logistic loss we use the Ipopt solver [16].”
MethodAdvances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,846
A Generalized Representer Theorem
Schölkopf, B., Herbrich, R., & Smola, A. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_27 · Cited by 1,589 · cited 1×
“Making use of the representer theorem [13], we can now express weight vector w as a linear combination of the mapped training instances; that is,”
MethodDataset Shift in Machine Learning
(2008) · doi:10.7551/mitpress/9780262170055.001.0001 · Cited by 1,553
An overview of bilevel optimization
Colson, B., Marcotte, P., & Savard, G. (2007) · Annals of Operations Research · doi:10.1007/s10479-007-0176-2 · Cited by 1,409 · cited 1×
“Numerous approaches that address bilevel programs have been studied, for instance, based on gradient descent, penalty function, and trust-region methods; see, for instance, [2] for a detailed survey.”
BackgroundThe polynomial hierarchy and a simple model for competitive analysis
Jeroslow, R. G. (1985) · Mathematical Programming · doi:10.1007/bf01586088 · Cited by 425 · cited 2×
“Even the simplest instance in which all constraints and objectives are linear is known to be NP-hard [8].”
BackgroundNightmare at test time: robust learning by feature deletion
Globerson, A., & Roweis, S. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143889 · Cited by 309 · cited 1×
“Similarly, worstcase solutions to classification games in which the adversary deletes input features or performs arbitrary feature transformation have been studied [3, 6, 7, 14, 4].”
Background(2000) · Applied Physics Letters · doi:10.1162/153244303321897726 · Cited by 157
Learning to classify with missing and corrupted features
Dekel, O., Shamir, O., & Xiao, L. (2009) · Machine Learning · doi:10.1007/s10994-009-5124-8 · Cited by 139 · cited 1×
“Similarly, worstcase solutions to classification games in which the adversary deletes input features or performs arbitrary feature transformation have been studied [3, 6, 7, 14, 4].”
Background
Cited by
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2017) · CINECA IRIS Institutial Research Information System (University of Genoa) · Cited by 1,872
Evasion Attacks against Machine Learning at Test Time
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40994-3_25 · Cited by 890
“Under certain conditions, such problems can be solved using a Nash equilibrium approach [7, 8].”
MethodSoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502
“ifier) and the adversary responds optimally (choose a least cost evasion). Stackelberg games also allow for scalability compared to the corresponding simultaneous move game [90], [91]. Bruckner et al. [92] first recognized that test data manipulation can be seen in the PAC framework as modifying the distribution D to :…”
BackgroundCertified Defenses for Data Poisoning Attacks
Steinhardt, J., Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1706.03691 · Cited by 479
“Bilevel optimization has been considered in the related setting of Stackelberg games (Brückner and Scheffer, 2011; Brückner et al., 2012; Zhou and Kantarcioglu, 2016), and is natural to apply here as well.”
MethodA Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View
Liu, Q., Li, P., Zhao, W., Cai, W., Yu, S., & Leung, V. C. M. (2018) · IEEE Access · doi:10.1109/access.2018.2805680 · Cited by 421
“[86] M. Brückner, C. Kanzow, and T. Scheffer, ‘‘Static prediction games for adversarial learning problems,’’ J. Mach.”
MethodProtecting location privacy: optimal strategy against localization attacks
Shokri, R., Theodorakopoulos, G., Troncoso, C., Hubaux, J.-P., & Le Boudec, J.-Y. (2012) · Proceedings of the 2012 ACM conference on Computer and communications security · doi:10.1145/2382196.2382261 · Cited by 385
Optimization problems for machine learning: A survey
Gambella, C., Ghaddar, B., & Naoum-Sawaya, J. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.08.045 · Cited by 365
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning AttacksPreprint
Suciu, O., Mărginean, R., Kaya, Y., Daumé, H., & Dumitraş, T. (2018) · arXiv (Cornell University) · Cited by 332
On the (Statistical) Detection of Adversarial Examples
Grosse, K., Manoharan, P., Papernot, N., Michael Backes, A., & McDaniel, P. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06280 · Cited by 330
“Other approaches make use of game theory [6, 9, 21].”
MethodFairness is not static: deeper understanding of long term fairness via simulation studies
D'Amour, A., Srinivasan, H., Atwood, J., Baljekar, P., Sculley, D., & Halpern, Y. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372878 · Cited by 156
“This setting is a special case of a sequential two-player game called the Stackleberg game [2].”
BackgroundInherent Tradeoffs in the Fair Determination of Risk Scores
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603197 · Cited by 310
The Social Cost of Strategic Classification
Milli, S., Miller, J., Dragan, A. D., & Hardt, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287576 · Cited by 138
“Brückner and Scheffer [5], Dong et al. [12], Hardt et al. [13] give algorithms to compute the Stackelberg equilibrium, which corresponds to the extreme τ ∗ solution in our trade-off curves.”
BackgroundInfluentialStrategic Classification from Revealed Preferences
Dong, J., Roth, A., Schutzman, Z., Waggoner, B., & Wu, Z. S. (2018) · Proceedings of the 2018 ACM Conference on Economics and Computation · doi:10.1145/3219166.3219193 · Cited by 109
“1Brückner and Scheffer [8] model a single data generation player who manipulates the data distribution, and experiences cost equal to the squared l2 distance of his manipulation.”
BackgroundMethodResultInfluentialA taxonomy and survey of attacks against machine learning
Pitropakis, N., Panaousis, E., Giannetsos, T., Anastasiadis, E., & Loukas, G. (2019) · Computer Science Review · doi:10.1016/j.cosrev.2019.100199 · Cited by 263
“…al. [41] Wang et al. [42] Demetrio et al. [43] Table 3: Comparative analysis of Intrusion Detection Systems’ Attack Evaluation Bruckner and Scheffer [47] in the context of spam email filtering study Stack-elberg games in which the learner acts as the leader committing to a predictive model, while…”
BackgroundStatic prediction games for adversarial learning problems
Brückner, M., Kanzow, C., & Scheffer, T. (2012) · publish.UP (University of Potsdam) · doi:10.5555/2503308.2503326 · Cited by 240
“Stackelberg equilibria of adversarial classification problems can be identified by solving a bilevel optimization problem (Brückner and Scheffer, 2011).”
BackgroundMethodInfluentialSelf-citationThe Disparate Effects of Strategic Manipulation
Hu, L., Immorlica, N., & Vaughan, J. W. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287597 · Cited by 134
“As in previous work in strategic classication, we cast the problem as a Stackelberg game in which the learner moves rst and publishes her classier before candidates best-respond and manipulate their features [1, 5, 7, 14].”
BackgroundMethodResultInfluentialHow Do Classifiers Induce Agents to Invest Effort Strategically?
Kleinberg, J., & Raghavan, M. (2019) · Proceedings of the 2019 ACM Conference on Economics and Computation · doi:10.1145/3328526.3329584 · Cited by 105
“in the case of spam detection, and more recently in terms of Stackelberg competitions, in which the evaluator publishes a rule and the agent may respond by manipulating their features strategically [5, 13, 18, 23, 29].”
BackgroundHardt, M., Megiddo, N., Papadimitriou, C., & Wootters, M. (2016) · Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science · doi:10.1145/2840728.2840730 · Cited by 223
Data Poisoning Attacks against Autoregressive Models
Alfeld, S., Zhu, X., & Barford, P. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10237 · Cited by 205
“A somewhat orthogonal line of research has posed the problem of learning in the presence of adversaries in a game theoretic context (Liu and Chawla 2009; Br¨uckner, Kanzow, and Scheffer 2012; Dalvi et al. 2004; Br¨uckner and Scheffer 2009; 2011).”
BackgroundDetecting Adversarial Image Examples in Deep Neural Networks with Adaptive Noise Reduction
Liang, B., Li, H., Su, M., Li, X., Shi, W., & Wang, X. (2021) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2018.2874243 · Cited by 196
“Many countermeasures against evasion attacks have been proposed, such as using game theory [63], [64] or probabilistic models [65], [66] to predict attack strategy to construct more robust classifiers, employing multiple classifier systems (MCSs) [67], [68], [69] to increase the difficulty of evasion, and optimizing f…”
Vorobeychik, Y., & Kantarcıoğlu, M. (2018) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00861ed1v01y201806aim039 · Cited by 194
Towards the Science of Security and Privacy in Machine Learning
Papernot, N., McDaniel, P. D., Sinha, A., & Wellman, M. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03814 · Cited by 192
“Bruckner et al. [92] first recognized that test data manipulation can be seen in the PAC framework as modifying the distribution D to .”
Privacy Games: Optimal User-Centric Data Obfuscation
Shokri, R. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0024 · Cited by 176
“Our problem is also related to the problem of adversarial machine learning [5, 30] and the design of security mechanisms, such as intelligent spam detection algorithms [13, 35, 37], against adaptive attackers.”
BackgroundVorobeychik, Y., & Kantarcıoğlu, M. (2018) · Synthesis lectures on artificial intelligence and machine learning · doi:10.1007/978-3-031-01580-9 · Cited by 176
Adversarial XAI Methods in Cybersecurity
Kuppa, A., & Le-Khac, N.-A. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3117075 · Cited by 141
Addressing Adversarial Attacks Against Security Systems Based on Machine Learning
Apruzzese, G., Colajanni, M., Ferretti, L., & Marchetti, M. (2019) · doi:10.23919/cycon.2019.8756865 · Cited by 161
“An example of application to spam detection is described in [46].”
MethodOperationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157
“Evasion attacks can be protected against by devising systems composed of multiple classifiers, as outlined in [137], or other adversary-aware learning algorithms (e.g. one-and-a-half-class multiple classifiers) as outlined in [135].”
BackgroundOn the Security of Machine Learning in Malware C&C Detection: A Survey
Gardiner, J., & Nagaraja, S. (2017) · ACM Computing Surveys · doi:10.1145/3003816 · Cited by 121
Stochastic Hyperparameter Optimization through Hypernetworks
Lorraine, J., & Duvenaud, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09419 · Cited by 76
“Games where learning a best-response can be applied include adversarial training (Goodfellow et al., 2014), or Stackelberg competitions (e.g., Brückner & Scheffer (2011)).”
BackgroundAdversarial support vector machine learning
Zhou, Y., Kantarcıoğlu, M., Thuraisingham, B. M., & Xi, B. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339697 · Cited by 143
“Brückner and Scheffer model adversarial prediction problems as Stackelberg games [5].”
BackgroundBridging Machine Learning and Mechanism Design towards Algorithmic Fairness
Finocchiaro, J., Maio, R., Monachou, F., Patro, G. K., Raghavan, M., Stoica, A.-A., & Tsirtsis, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445912 · Cited by 78
“cation [42, 43, 55, 65, 70, 104, 112] which focuses on incentive-aware machine learning algorithms that try to reduce misclassification caused by transparency-induced strategic behavior.”
BackgroundFeature Cross-Substitution in Adversarial Classification
Li, B., & Vorobeychik, Y. (2014) · neural information processing systems · Cited by 111
Learning Strategy-Aware Linear ClassifiersPreprint
Chen, Y., Liu, Y., & Podimata, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.04004 · Cited by 111
“A byproduct of this is that our solution concept is the one of Stackelberg regret, contrary to the notions of Nash and Stackelberg Equilibria considered in [36, 15, 30].”
BackgroundMethodInfluentialPrivacy Games Along Location Traces: A Game-Theoretic Framework for Optimizing Location Privacy
Shokri, R., Theodorakopoulos, G., & Troncoso, C. (2016) · ACM Transactions on Privacy and Security · doi:10.1145/3009908 · Cited by 110
Performative Prediction: Past and Future
Hardt, M., & Mendler-Dünner, C. (2025) · Statistical Science · doi:10.1214/25-sts986 · Cited by 105
Chen, L., Hou, S., & Ye, Y. (2017) · Annual Computer Security Applications Conference · doi:10.1145/3134600.3134636 · Cited by 105
“, well-crafted attack as it shows most effective) by comparisons with other popular defense methods, including (1) feature evenness (denoted as Defense1) which enables the Original-Classifier to learn more evenlydistributed feature weights using the method proposed in [23]; (2) classifier retraining (denoted as Defens…”
MethodA System-Driven Taxonomy of Attacks and Defenses in Adversarial Machine Learning
Sadeghi, K., Banerjee, A., & Gupta, S. K. S. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2020.2968933 · Cited by 103
“In game theoretical strategies, on one hand, the adversary attempts to generate an adversarial sample to be classified as the target class data, and on the other hand, the ML application attempts to recognize the sample and reject it [42], [195], [196].”
BackgroundPATTERN RECOGNITION SYSTEMS UNDER ATTACK: DESIGN ISSUES AND RESEARCH CHALLENGES
BIGGIO, B., FUMERA, G., & ROLI, F. (2014) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001414600027 · Cited by 79
“[31]; then, more rigorous approaches have introduced Nash and Stackelberg games for secure learning, deriving formal conditions for the corresponding equilibrium strategies [25, 45, 49, 26, 24].”
BackgroundA review of attacker-defender games: Current state and paths forward
Hunt, K., & Zhuang, J. (2023) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.04.009 · Cited by 98
“In this domain, Stackleberg games have been proposed to study the strategic interactions between the learner (e.g., machine learning model) and attacker (e.g., data generator) (Brückner & Scheffer, 2011).”
Approaches to adversarial drift
Kantchelian, A., Afroz, S., Huang, L., Islam, A. C., Miller, B., Tschantz, M. C., Greenstadt, R., Joseph, A. D., & Tygar, J. D. (2013) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/2517312.2517320 · Cited by 96
“Finally, we mention the game theoretic line of work [5, 6, 10, 25].”
MethodCracking Classifiers for Evasion: A Case Study on the Google's Phishing Pages Filter
Liang, B., Su, M., You, W., Shi, W., & Yang, G. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883060 · Cited by 64
“Game-theoretical approaches [22][23] model the interactions between the adversary and the classifier as a game.”
BackgroundCausal Strategic Linear RegressionPreprint
Shavit, Y., Edelman, B., & Axelrod, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10066 · Cited by 88
“Our use of a Euclidean squared norm cost function is shared by the first paper in a strategic classfication setting (Brückner & Scheffer, 2011).”
MethodRandomization matters. How to defend against strong adversarial attacksPreprint
Pinot, R., Ettedgui, R., Rizk, G., Chevaleyre, Y., & Atif, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11565 · Cited by 85
“For example Brückner & Scheffer (2011) views adversarial example attacks and defenses as a Stackelberg game.”
Randomized Prediction Games for Adversarial Machine Learning
Bulò, S. R., Biggio, B., Pillai, I., Pelillo, M., & Roli, F. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2593488 · Cited by 83
A survey of game theoretic approach for adversarial machine learning
Zhou, Y., Kantarcıoğlu, M., & Xi, B. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1259 · Cited by 79
“Bruckner and Scheffer (2011) study this form of adversarial learning problems and present equilibrium solutions to the corresponding Stackelberg classification games.”
BackgroundLarge-Scale Identification of Malicious Singleton Files
Li, B., Roundy, K., Gates, C., & Vorobeychik, Y. (2017) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3029806.3029815 · Cited by 78
“volume of singleton malware is largely the byproduct of adaptations to anti-virus technology [1, 17, 2].”
BackgroundA Survey of Game Theoretic Approaches for Adversarial Machine Learning in Cybersecurity Tasks
Dasgupta, P., & Collins, J. B. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2847 · Cited by 76
Adversarial Detection with Model Interpretation
Liu, N., Yang, H., & Hu, X. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220027 · Cited by 56
“To estimate feature robustness, a straightforward way is to find the closed-form formula for each feature and explicitly compute the difficulty of changing its value [17, 53], or to unify the cost to a constant value [11].”
BackgroundMethodAhmadi, S., Beyhaghi, H., Blum, A. L., & Naggita, K. (2021) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3465456.3467629 · Cited by 71
“In machine learning, strategic classification deals with the problem of learning a classifier when the learner relies on data that is provided by strategic agents [4, 10].”
BackgroundManipulation-Robust Prediction
Björkegren, D., Blumenstock, J. E., & Knight, S. (2026) · American Economic Review · doi:10.1257/aer.20241087 · Cited by 71
Decisions, Counterfactual Explanations and Strategic BehaviorPreprint
Tsirtsis, S., & Gomez-Rodriguez, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04333 · Cited by 70
“Similarly as in our work, previous work on strategic machine learning also assumes that individuals may use knowledge, gained by transparency, to invest effort strategically in order to receive either a positive prediction (Dalvi et al., 2004; Brückner and Scheffer, 2011; Hardt et al., 2016; Dong et al., 2018; Hu et a…”
MethodAlgorithmic Contract Theory: A Survey
Dütting, P., Feldman, M., & Talgam-Cohen, I. (2024) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000113 · Cited by 70
Curie: A method for protecting SVM Classifier from Poisoning Attack
Laishram, R., & Phoha, V. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01584 · Cited by 32
“Most of the works [8,7] on secure classifier algorithms uses a game theory model - the problem is modeled as a game between the adversary and the learner.”
BackgroundMitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603203 · Cited by 65
Alternative Microfoundations for Strategic ClassificationPreprint
Jagadeesan, M., Mendler-Dünner, C., & Hardt, M. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2106.12705 · Cited by 66
“Existing work on strategic classification has mostly followed standard microfoundations for modeling agent behavior in response to a decision rule, e.g., (Dalvi et al., 2004; Brückner and Scheffer, 2011; Hardt et al., 2016a; Khajehnejad et al.; Tsirtsis and Gomez-Rodriguez, 2020) to name a few.”
Optimal randomized classification in adversarial settings
Vorobeychik, Y., & Li, B. (2014) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2615731.2615811 · Cited by 64
“The study of machine learning in such settings has come to be known as adversarial machine learning [8, 16, 3, 5, 15, 2, 6, 14, 10, 7, 18].”
BackgroundA Game-Theoretic Analysis of Adversarial Classification
Dritsoula, L., Loiseau, P., & Musacchio, J. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2718494 · Cited by 62
“To the best of our knowledge, the only works that analyze adversarial classification using nonzero-sum games are the studies by Br¨uckner et al. [11]–[13], and the studies by Lis´y et al. [29] and Samusevich [45].”
ResultOptimal Decision Making Under Strategic Behavior
Tsirtsis, S., Tabibian, B., Khajehnejad, M. A., Singla, A. K., Schölkopf, B., & Gomez-Rodriguez, M. (2024) · Management Science · doi:10.1287/mnsc.2021.02567 · Cited by 61
“With this motivation, there has been a recent flurry of work on strategic classification [1, 2, 5, 6, 8, 11, 16].”
BackgroundAutomatically Inferring Malware Signatures for Anti-Virus Assisted Attacks
Wressnegger, C., Freeman, K., Yamaguchi, F., & Rieck, K. (2017) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3052973.3053002 · Cited by 60
“While some of these approaches provide excellent detection capabilities and runtime performance, the application of machine learning can induce additional security problems [17, 41, 48, 59] whose elimination is an ongoing research effort [7, 23, 61].”
BackgroundExplicit Defense Actions Against Test-Set Attacks
Alfeld, S., Zhu, X., & Barford, P. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10767 · Cited by 58
“A separate line of research has posed the problem of learning in the presence of adversaries in game theoretic contexts (Liu and Chawla 2009; Brückner, Kanzow, and Scheffer 2012; Dalvi et al. 2004; Brückner and Scheffer 2009; 2011; Hardt et al. 2016; Brückner and Scheffer 2011).”
BackgroundScalable Optimization of Randomized Operational Decisions in Adversarial Classification Settings
Li, B., & Vorobeychik, Y. (2015) · International Conference on Artificial Intelligence and Statistics · Cited by 57
Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved FeaturesPreprint
Tong, L., Li, B., Hajaj, C., Xiao, C., Zhang, N., & Vorobeychik, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.08327 · Cited by 57
“…leveraging traditional machine learning techniques, such as regularization [35], [36], [23], [28], to game theoretic approaches in which the defender chooses a classifier that the attacker attempts to evade [7], [4], [18], [39], [5], to retraining with adversarial evasion examples [19], [15], [16].”
BackgroundDefending Jailbreak Prompts via In-Context Adversarial Game
Zhou, Y., Han, Y., Zhuang, H., Guo, K., Liang, Z., Bao, H., & Zhang, X. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1121 · Cited by 56
Xi, B. (2020) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.1511 · Cited by 55
“(Br¨uckner & Scheffer, 2011) showed that the Stackelberg game with the worst case loss given the defender chose a hinge loss led to a SVM for invari-ances, and a Stackelberg game with linear loss led to a regular SVM.”
BackgroundInfluentialAdversarial Example GamesPreprint
Bose, A. J., Gidel, G., Berard, H., Cianflone, A., Vincent, P., Lacoste-Julien, S., & Hamilton, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00720 · Cited by 55
“Such an application falls in line with other general approaches to red teaming AI systems [10] and verifiability in AI development.”
ResultAdversarial Machine Learning: Bayesian Perspectives
Insua, D. R., Naveiro, R., Gallego, V., & Poulos, J. (2023) · Journal of the American Statistical Association · doi:10.1080/01621459.2023.2183129 · Cited by 50
“In Stackelberg prediction games , Brückner and Scheffer (2011) assume full information of the attacker about the predictive model used by the defender who, in addition, has full information about the adversary’s costs and action space.”
Background(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603207 · Cited by 45
Suciu, O., Mărginean, R., Kaya, Y., Daumé, H., & Dumitraş, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.06975 · Cited by 48
“Furthermore, [27, 7] introduce game theoretical Stackelberg formulations for the interaction between the adversary and the data miner in the case of data manipulations.”
BackgroundStrategic Classification in the DarkPreprint
Ghalme, G. S., Nair, V., Eilat, I., Talgam-Cohen, I., & Rosenfeld, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.11592 · Cited by 48
“This line of research was initiated by (Brückner & Scheffer, 2011; Hardt et al., 2016), and has quickly amassed a multitude of insightful follow-up works (see below).”
MethodThe Challenges in ML-Based Security for SDN
Nguyen, T. N. (2018) · Cyber Security in Networking Conference (CSNet) · doi:10.1109/csnet.2018.8602680 · Cited by 46
“Another way to attack was described by Bruckner [41] as a single Stackelberg Prediction Game (SPG).”
BackgroundA Stackelberg game for distributed formation of business-driven services communities
Wahab, O. A., Bentahar, J., Otrok, H., & Mourad, A. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.09.047 · Cited by 45
Security Analytics: Essential Data Analytics Knowledge for Cybersecurity Professionals and Students
Verma, R. M., Kantarcıoğlu, M., Marchette, D. J., Leiss, E. L., & Solorio, T. (2015) · IEEE Security & Privacy · doi:10.1109/msp.2015.121 · Cited by 44
A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial ExamplesPreprint
Wang, B., Gao, J., & Qi, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00334 · Cited by 45
“…assuming a polynomial number of membership queries (Lowd & Meek, 2005), formalizing the attack process as a two-person sequential Stackelberg game (Brückner & Scheffer, 2011; Liu & Chawla, 2010), a min-max strategy (training a classifier with best performance under the worst perturbation) (Dekel…”
BackgroundAdding Robustness to Support Vector Machines Against Adversarial Reverse Engineering
Alabdulmohsin, I., Gao, X., & Zhang, X. (2014) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2661829.2662047 · Cited by 43
“Several lines of research have used such game-theoretic setting [14, 15, 21, 33].”
BackgroundMethodA Game-Theoretic Approach to Design Secure and Resilient Distributed Support Vector Machines
Zhang, R., & Zhu, Q. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2802721 · Cited by 43
Deception in Finitely Repeated Security Games
Nguyen, T. H., Wang, Y., Sinha, A., & Wellman, M. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012133 · Cited by 43
A Game Theoretic Analysis of Additive Adversarial Attacks and DefensesPreprint
Pal, A., & Vidal, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06530 · Cited by 40
“[3, 2] study Nash Equilibria for a similar setting, where the defender is now a learner who has to output the weights of a classifier that predicts which distribution the input was sampled from.”
BackgroundSoK: Applying Machine Learning in Security - A SurveyPreprint
Jiang, H., Nagra, J., & Ahammad, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03186 · Cited by 39
“[46] of 2010 highlighted use cases and challenges for ML in network intrusion detection, but did not incorporate a high-level review of ML in security in recent years.”
BackgroundRobust Federated Learning via Collaborative Machine Teaching
Han, Y., & Zhang, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5826 · Cited by 38
“Finally, previous machine teaching methods suffer from intense computation for solving a bi-level stackelberg game (Br¨uckner and Scheffer 2011) with non-linear programming or combinatorial optimization techniques.”
MethodSparse Feature Attacks in Adversarial Learning
Yin, Z., Wang, F., Liu, W., & Chawla, S. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2790928 · Cited by 38
On Sparse Feature Attacks in Adversarial Learning
Wang, F., Liu, W., & Chawla, S. (2014) · Proceedings · doi:10.1109/icdm.2014.117 · Cited by 37
“[10] also made the unstated assumption that the adversary has the ability to change all the features i.”
BackgroundMethodInfluentialOn Classification of Strategic Agents Who Can Both Game and Improve
Ahmadi, S., Beyhaghi, H., Blum, A. L., & Naggita, K. (2022) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.forc.2022.3 · Cited by 37
Convex Adversarial Collective Classification
Torkamani, M., & Lowd, D. (2013) · International Conference on Machine Learning · Cited by 36
Linear Classifiers that Encourage Constructive AdaptationPreprint
Chen, Y., Wang, J., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.00355 · Cited by 36
“We use a two player Stackelberg game model [10, 25] to capture the interaction between the agents and the mechanism designer, and show that under some assumptions, the best response of an agent has a closed-form solution, which allows us to solve the mechanism design problem.”
MethodSoK: Machine Learning GovernancePreprint
Chandrasekaran, V., Jia, H., Thudi, A., Travers, A., Yaghini, M., & Papernot, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.10870 · Cited by 36
Imitative Attacker Deception in Stackelberg Security Games
Nguyen, T. H., & Xu, H. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/75 · Cited by 35
“Research on adversarial learning has attempted to design various types of attacks to machine learning algorithms [Brückner et al., 2012; Brückner and Scheffer, 2011; Barreno et al., 2010; 2006; Lowd and Meek, 2005] The attacker deception in our game can be viewed as a type of causative attack to…”
BackgroundBackward induction algorithm for a class of closed-loop Stackelberg games
Kicsiny, R., Varga, Z., & Scarelli, A. (2014) · European Journal of Operational Research · doi:10.1016/j.ejor.2014.02.057 · Cited by 35
“Stackelberg games are important in the field of optimization theory as well, since they can be considered as bi-level optimization problems (Brückner & Scheffer, 2011; Dempe & Franke, 2014; Du, Jiao, & Chen, 2014; Jungers, Trélat, & Abou-Kandil, 2011; Motto, 2005; Nie, 2005; Nie et al., 2006).”
BackgroundEvasion-Robust Classification on Binary Domains
Li, B., & Vorobeychik, Y. (2018) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3186282 · Cited by 34
“Indeed, the issue of only being able to compute an attack heuristically with respect to a model is a fundamental problem in most prior adversarial evasion modeling efforts, excepting several which make strong assumptions about adversarial cost and loss functions (e.g., Brückner and Scheffer (2011)).”
BackgroundMethodInfluentialA General Retraining Framework for Scalable Adversarial ClassificationPreprint
Li, B., Vorobeychik, Y., & Chen, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.02606 · Cited by 33
“A number of approaches have been proposed for making learning algorithms more robust to adversarial evasion attacks [22, 10, 11, 12, 14].”
BackgroundTechnologies for Trustworthy Machine Learning: A Survey in a Socio-Technical ContextPreprint
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., González-Zelaya, V., Missier, P., Ng, M., & van Moorsel, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08911 · Cited by 33
“[27] improve [26] with a randomized strategy selection in their prediction system, thus improving the trade–off options between attack detection and false alarm.”
BackgroundMethodPAC-Learning for Strategic ClassificationPreprint
Sundaram, R., Vullikanti, A. K. S., Xu, H., & Yao, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03310 · Cited by 30
“In fact, restricted special cases of this model have been considered in previous works [12, 26, 19], among which two of them are most”
BackgroundMethodAn Exploratory Study of Cognitive Sciences Applied to Cybersecurity
Andrade, R., Fuertes, W., Cazares, M. F., Ortiz-Garcés, I., & Navas, G. (2022) · Electronics · doi:10.3390/electronics11111692 · Cited by 29
“Researchers have identified that the possible actions or strategies of adversaries can be studied using game theory models with incomplete information based on Stackelberg’s proposals [8].”
BackgroundLearning Losses for Strategic Classification
Lechner, T., & Urner, R. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20696 · Cited by 29
Less Is More: Robust and Novel Features for Malicious Domain Detection
Hajaj, C., Hason, N., & Dvir, A. Z. (2022) · Electronics · doi:10.3390/electronics11060969 · Cited by 29
Modeling Adversarial Learning as Nested Stackelberg Games
Zhou, Y., & Kantarcıoğlu, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-31750-2_28 · Cited by 29
Statistics and Data Science for Cybersecurity
Hero, A. O., Kar, S., Moura, J. M. F., Neil, J., Poor, H. V., Turcotte, M., & Xi, B. (2023) · Harvard Data Science Review · doi:10.1162/99608f92.a42024d0 · Cited by 28
The Role of Randomness and Noise in Strategic ClassificationPreprint
Braverman, M., & Garg, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.08377 · Cited by 28
On the Actionability of Outcome Prediction
Liu, L. T., Barocas, S., Kleinberg, J. M., & Levy, K. E. C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30229 · Cited by 27
Evaluation of random forest classifier in security domain
Khorshidpour, Z., Hashemi, S., & Hamzeh, A. (2017) · Applied Intelligence · doi:10.1007/s10489-017-0907-2 · Cited by 27
“in [10] modeled the interaction between a classifier and an adversary as a Stackelberg competition in which the classifier plays the role of the leader and the adversary may react according to the leader’s move.”
BackgroundThe Role of Randomness and Noise in Strategic Classification
Braverman, M., & Garg, S. (2020) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.forc.2020.9 · Cited by 27
Metadata
- Authors
- Brückner, M., & Scheffer, T.
- Affiliations
- University of Potsdam (DE)
- ORCID
- Scheffer, Tobias
- Venue
- Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2011-08-21
- Pages
- 547-555
- OpenAlex
- W1966912382
- Cited by
- 229
- References
- 19
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 11 · 2023: 32 · 2022: 9 · 2021: 29 · 2020: 25 · 2019: 16 · 2018: 28 · 2017: 16 · 2016: 21 · 2015: 7 · 2014: 8 · 2013: 6 · 2012: 6
- FWCI
- 4.80
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Keywords
- Stackelberg game, email spam detection, game-theoretic optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.