Respite for SMEs: A Systematic Review of Socio-Technical Cybersecurity Metrics
Open accessThe DOI matches its Crossref record; 135 of the 136 references it cites were found in a registry or an index.
Abstract
Cybersecurity threats are on the rise, and small- and medium-sized enterprises (SMEs) struggle to cope with these developments. To combat threats, SMEs must first be willing and able to assess their cybersecurity posture. Cybersecurity risk assessment, generally performed with the help of metrics, provides the basis for an adequate defense. Significant challenges remain, however, especially in the complex socio-technical setting of SMEs. Seemingly basic questions, such as how to aggregate metrics and ensure solution adaptability, are still open to debate. Aggregation and adaptability are vital topics to SMEs, as they require the assimilation of metrics into an actionable advice adapted to their situation and needs. To address these issues, we systematically review socio-technical cybersecurity metric research in this paper. We analyse aggregation and adaptability considerations and investigate how current findings apply to the SME situation. To ensure that we provide valuable insights to researchers and practitioners, we integrate our results in a novel socio-technical cybersecurity framework geared towards the needs of SMEs. Our framework allowed us to determine a glaring need for intuitive, threat-based cybersecurity risk assessment approaches for the least digitally mature SMEs. In the future, we hope our framework will help to offer SMEs some deserved respite by guiding the design of suitable cybersecurity assessment solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


![Figure 3 Asocio-technical system embedded within an external environment, based on Davis et al. [7].](https://figures.cochat.ai/paper-figures/c2803feea2b44a18814b822d58a63eb2/respite-for-smes-a-systematic-review-of-socio-technical-cybersecurity-metrics-figure-3.png)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Several systematic reviews using ASReview have already been published. For some examples, see [56-60].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · cited 3×
“…broad coverage of the cybersecurity metrics field, we employed a novel systematic review methodology blending active learning and snowballing (SYMBALS, [58]), which combines existing methods into a swift and accessible methodology, while following authoritative systematic review guidelines [59–61].”
BackgroundMethodLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 2×
“From a set of included papers, a researcher can find additional relevant papers by consulting references (backward snowballing) and citations (forward snowballing) [63].”
BackgroundMethodBrereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394 · cited 1×
“Criterion E9 requires the consideration of the full text to be determined, as abstracts do not contain enough information to make a decision regarding this intricate topic [67].”
BackgroundLearning in the Presence of Concept Drift and Hidden Contexts
Widmer, G., & Kubat, M. (1996) · Machine Learning · doi:10.1023/a:1018046501280 · Cited by 1,857 · cited 2×
“We looked towards existing research to determine interesting dimensions of adaptability, such as missing or dirty data [56] and concept drift [57].”
BackgroundFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“STS research has uncovered the dangers of considering social and technical elements separately [23] and has offered insight into how to avoid these dangers [7].”
BackgroundAn open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157 · cited 5×
“We believe in the construct validity of our systematic review methodology SYM-BALS [58], as it is based on widely-accepted methods [62,63] and guidelines [59–61].”
BackgroundMethodInfluentialHerath, T., & Rao, H. R. (2009) · Decision Support Systems · doi:10.1016/j.dss.2009.02.005 · Cited by 857 · cited 1×
“PMT reserves a prominent role for extrinsic motivators and threat appraisal [26].”
BackgroundManadhata, P. K., & Wing, J. M. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.60 · Cited by 625 · cited 1×
“Digital enablers should additionally be aware of internal and external dependencies [92], allowing them to specify their attack surface [101].”
BackgroundShin, Y., Meneely, A., Williams, L., & Osborne, J. A. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.81 · Cited by 607 · cited 2×
“In papers covering the defender, it is quite common to ignore threats entirely [78] or to use a proxy such as the prevalence of vulnerabilities to represent threats [79].”
BackgroundA review of cyber security risk assessment methods for SCADA systems
Cherdantseva, Y., Burnap, P., Blyth, A., Eden, P., Jones, K., Soulsby, H., & Stoddart, K. (2016) · Computers & Security · doi:10.1016/j.cose.2015.09.009 · Cited by 606 · cited 1×
“…et al. [39] 2020 Moving Target Defense × Liang and Xiao [40] 2013 Network Security × Ramos et al. [41] 2017 Network Security Cherdantseva et al. [42] 2016 SCADA Systems Morrison et al. [43] 2018 Software Security × He et al. [44] 2019 Unknown Vulnerabilities × Xie et al. [45] 2019 Wireless…”
BackgroundAdvancing socio-technical systems thinking: A call for bravery
Davis, M. C., Challenger, R., Jayewardene, D. N. W., & Clegg, C. W. (2014) · Applied Ergonomics · doi:10.1016/j.apergo.2013.02.009 · Cited by 458 · cited 7×
“The view of STS is that joint consideration of social and technical elements is necessary [7].”
BackgroundMethodInfluentialHarrison, H., Griffin, S. J., Kuhn, I., & Usher-Smith, J. A. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-0897-3 · Cited by 449 · cited 1×
“Many other active learning tools exist that are worth considering [66].”
BackgroundA survey of cyber security management in industrial control systems
Knowles, W., Prince, D., Hutchison, D., Disso, J. F. P., & Jones, K. (2015) · International Journal of Critical Infrastructure Protection · doi:10.1016/j.ijcip.2015.02.002 · Cited by 344 · cited 1×
“…and Bridges [32] 2020 Cyber Defense × Kordy et al. [33] 2014 Directed Acyclic Graphs × Cadena et al. [34] 2020 Incident Management Knowles et al. [35] 2015 Industrial Control Systems Asghar et al. [36] 2019 Industrial Control Systems Eckhart et al. [37] 2019 Industrial Control Systems × Jing et…”
BackgroundDAG-based attack and defense modeling: Don’t miss the forest for the attack trees
Kordy, B., Piètre-Cambacédès, L., & Schweitzer, P. (2014) · Computer Science Review · doi:10.1016/j.cosrev.2014.07.001 · Cited by 321 · cited 2×
“…2016 Generic Cho et al. [18] 2019 Generic Husák et al. [31] 2019 Attack Prediction Iannacone and Bridges [32] 2020 Cyber Defense × Kordy et al. [33] 2014 Directed Acyclic Graphs × Cadena et al. [34] 2020 Incident Management Knowles et al. [35] 2015 Industrial Control Systems Asghar et al. [36]…”
BackgroundSurvey of Attack Projection, Prediction, and Forecasting in Cyber Security
Husak, M., Komarkova, J., Bou-Harb, E., & Celeda, P. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2871866 · Cited by 316 · cited 2×
“…Other RE WLC No Akinsanya et al. [133] Other RE WLC No Bharathi [134] Other AW WLC Yes Fertig et al. [135] Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al. [136] Other AW, KN, RE WLC No Cadena et al. [34] Other AW None No Wirtz and Heisel [137] Other AW, KN, AB, RE WLC,…”
BackgroundKim, W., Choi, B.-J., Hong, E.-K., Kim, S.-K., & Lee, D. (2003) · Data Mining and Knowledge Discovery · doi:10.1023/a:1021564703268 · Cited by 301 · cited 2×
“We looked towards existing research to determine interesting dimensions of adaptability, such as missing or dirty data [56] and concept drift [57].”
BackgroundGame Theory for Network Security
Liang, X., & Xiao, Y. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.062612.00056 · Cited by 293 · cited 2×
“Rudolph and Schwarz [30] 2012 Generic × Pendleton et al. [17] 2016 Generic Cho et al. [18] 2019 Generic Husák et al. [31] 2019 Attack Prediction Iannacone and Bridges [32] 2020 Cyber Defense × Kordy et al. [33] 2014 Directed Acyclic Graphs × Cadena et al. [34] 2020 Incident Management Knowles et al. [35] 2015 Industri…”
BackgroundA Survey on Systems Security Metrics
Pendleton, M., Garcia-Lebron, R., Cho, J.-H., & Xu, S. (2017) · ACM Computing Surveys · doi:10.1145/3005714 · Cited by 283 · cited 8×
“There is no standard definition of what constitutes a (cyber)security metric [17].”
BackgroundMethodInfluentialMenard, P., Bott, G. J., & Crossler, R. E. (2017) · Journal of Management Information Systems · doi:10.1080/07421222.2017.1394083 · Cited by 279 · cited 8×
“Threat perception lies at the core of PMT and is important in security applications using SDT [25].”
BackgroundMethodResultInfluentialA Survey of Moving Target Defenses for Network Security
Sengupta, S., Chowdhary, A., Sabur, A., Alshamrani, A., Huang, D., & Kambhampati, S. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2982955 · Cited by 269 · cited 1×
“…2019 Industrial Control Systems Eckhart et al. [37] 2019 Industrial Control Systems × Jing et al. [38] 2019 Internet Security × Sengupta et al. [39] 2020 Moving Target Defense × Liang and Xiao [40] 2013 Network Security × Ramos et al. [41] 2017 Network Security Cherdantseva et al. [42] 2016…”
BackgroundCorrelating human traits and cyber security behavior intentions
Gratian, M., Bandi, S., Cukier, M., Dykstra, J., & Ginther, A. (2018) · Computers & Security · doi:10.1016/j.cose.2017.11.015 · Cited by 266 · cited 1×
“This view has interesting implications in cybersecurity, where humans are generally found to be the weakest link [8,9].”
Background(2006) · Lecture Notes in Computer Science · doi:10.1007/11751540 · Cited by 260
pybliometrics: Scriptable bibliometrics using a Python interface to Scopus
Rose, M. E., & Kitchin, J. R. (2019) · SoftwareX · doi:10.1016/j.softx.2019.100263 · Cited by 228 · cited 1×
“The Python Scopus API wrapper “pybliomet-rics” [65] was used to retrieve all research available through the Scopus API that satisfied the query: The “AUTHKEY” field corresponds to the keywords that authors provided for a paper.”
MethodMulticriteria Decision Framework for Cybersecurity Risk Assessment and Management
Ganin, A. A., Quach, P., Panwar, M., Collier, Z. A., Keisler, J. M., Marchese, D., & Linkov, I. (2020) · Risk Analysis · doi:10.1111/risa.12891 · Cited by 207 · cited 1×
“…Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al. [136] Other AW, KN, RE WLC No Cadena et al. [34] Other AW None No Wirtz and Heisel [137] Other AW, KN, AB, RE WLC, WM No Ganin et al. [138] Enterprise (A) AW, DE, KN, AB, RE WLC No Luh et al. [90] Other AW, DE, AB, RE WLC Yes”
BackgroundDefining organisational information security culture—Perspectives from academia and industry
da Veiga, A., Astakhova, L. V., Botha, A., & Herselman, M. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101713 · Cited by 196 · cited 9×
“Advance cybersecurity knowledge and ability through clearly communicated and documented trainings [5,92,94].”
BackgroundInfluentialMourão, E., Pimentel, J. F., Murta, L., Kalinowski, M., Mendes, E., & Wohlin, C. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106294 · Cited by 191 · cited 1×
“Snowballing has proven to be a valuable addition to systematic reviews, even when reviews already include an extensive database search [64].”
BackgroundQuantified security is a weak hypothesis: a critical survey of results and assumptions
Verendel, V. (2009) · Proceedings of the 2009 workshop on New security paradigms workshop · doi:10.1145/1719030.1719036 · Cited by 180 · cited 2×
“Current paper 2021 Generic Verendel [29] 2009 Generic”
BackgroundMethodData Collection for Security Measurement in Wireless Sensor Networks: A Survey
Xie, H., Yan, Z., Yao, Z., & Atiquzzaman, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2883403 · Cited by 179 · cited 1×
“…et al. [41] 2017 Network Security Cherdantseva et al. [42] 2016 SCADA Systems Morrison et al. [43] 2018 Software Security × He et al. [44] 2019 Unknown Vulnerabilities × Xie et al. [45] 2019 Wireless Networks × Some exceptions are comprehensive and cover both social and technical factors [17,18].”
BackgroundFeng, N., Wang, H. J., & Li, M. (2014) · Information Sciences · doi:10.1016/j.ins.2013.02.036 · Cited by 177 · cited 4×
“Solutions can be found in this area in Bayesian networks [83–86] and analytic network process [50,51] techniques.”
BackgroundMethodInfluentialMoving from a ‘human-as-problem” to a ‘human-as-solution” cybersecurity mindset
Zimmermann, V., & Renaud, K. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.05.005 · Cited by 173 · cited 1×
“This puts employees in a position to be a part of the solution to SME cybersecurity challenges, rather than being the source of the challenges [98].”
BackgroundCybersecurity in industrial control systems: Issues, technologies, and challenges
Asghar, M. R., Hu, Q., & Zeadally, S. (2019) · Computer Networks · doi:10.1016/j.comnet.2019.106946 · Cited by 172 · cited 1×
“…[33] 2014 Directed Acyclic Graphs × Cadena et al. [34] 2020 Incident Management Knowles et al. [35] 2015 Industrial Control Systems Asghar et al. [36] 2019 Industrial Control Systems Eckhart et al. [37] 2019 Industrial Control Systems × Jing et al. [38] 2019 Internet Security × Sengupta et al.…”
BackgroundTaxonomy of compliant information security behavior
Padayachee, K. (2012) · Computers & Security · doi:10.1016/j.cose.2012.04.004 · Cited by 163 · cited 10×
“We would expect to see more focus from research on desire, and the related concept of motivation, based on the important role that motivation and internalisation play in SDT and PMT [24].”
BackgroundInfluentialVerizon: 2021 Data Breach Investigations Report
(2021) · Computer Fraud & Security · doi:10.1016/s1361-3723(21)00061-0 · Cited by 157
Calculated risk? A cybersecurity evaluation tool for SMEs
Benz, M., & Chatterjee, D. (2020) · Business Horizons · doi:10.1016/j.bushor.2020.03.010 · Cited by 144 · cited 7×
“Our framework was constructed based on earlier cybersecurity frameworks focusing on SMEs [10,15,92] or STS [5,91,93–95].”
BackgroundMethodInfluentialHeidt, M., Gerlach, J. P., & Buxmann, P. (2019) · Information Systems Frontiers · doi:10.1007/s10796-019-09959-1 · Cited by 143 · cited 4×
“Due to their lack of resources [3] and the complex socio-technical setting they operate in, SMEs struggle to address their cybersecurity issues autonomously [10].”
BackgroundInfluentialSocio-technical systems cybersecurity framework
Malatji, M., von Solms, S., & Marnewick, A. L. (2019) · Information and Computer Security · doi:10.1108/ics-03-2018-0031 · Cited by 141 · cited 9×
“In SME cybersecurity, the interplay between the social and the technical is essential [5], which is why SMEs are often studied from a socio-technical systems (STS) perspective [6].”
BackgroundMethodInfluentialA multidimensional approach to information security risk management using FMEA and fuzzy theory
Silva, M. M., de Gusmão, A. P. H., Poleto, T., Silva, L. C. E., & Costa, A. P. C. S. (2014) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2014.07.005 · Cited by 138 · cited 1×
“…AW WLC, WP, WM Yes Feng et al. [86] Enterprise (M/L) AW, RE BN Yes Manifavas et al. [77] Enterprise (A) AW, DE, KN, AB, RE WLC Yes Silva et al. [81] Other RE WLC, WP No Suhartana et al. [116] Enterprise (A) AB, RE WLC Yes Yadav and Dong [117] Other AW, KN, AB, RE None Yes Dehghanimohammadabadi…”
BackgroundDesign and validation of information security culture framework
Alhogail, A. A. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.03.054 · Cited by 134 · cited 6×
“This can be achieved through a culture of organisational commitment to cybersecurity [93,94].”
BackgroundMethodInfluentialSecurity Data Collection and Data Analytics in the Internet: A Survey
Jing, X., Yan, Z., & Pedrycz, W. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2863942 · Cited by 130 · cited 1×
“…Industrial Control Systems Asghar et al. [36] 2019 Industrial Control Systems Eckhart et al. [37] 2019 Industrial Control Systems × Jing et al. [38] 2019 Internet Security × Sengupta et al. [39] 2020 Moving Target Defense × Liang and Xiao [40] 2013 Network Security × Ramos et al. [41] 2017…”
BackgroundMartens, M., Wolf, R. P. D., & Marez, L. D. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.11.002 · Cited by 125 · cited 1×
“Humans are often considered the weakest link in cybersecurity [21].”
BackgroundModel-Based Quantitative Network Security Metrics: A Survey
Ramos, A. L., Lazar, M., Filho, R. H., & Rodrigues, J. J. P. C. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745505 · Cited by 121 · cited 2×
“…et al. [38] 2019 Internet Security × Sengupta et al. [39] 2020 Moving Target Defense × Liang and Xiao [40] 2013 Network Security × Ramos et al. [41] 2017 Network Security Cherdantseva et al. [42] 2016 SCADA Systems Morrison et al. [43] 2018 Software Security × He et al. [44] 2019 Unknown…”
BackgroundLo, C., & Chen, W.-J. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.07.015 · Cited by 117 · cited 7×
“Two examples of other possibilities are the use of analytic network process (ANP) techniques [50,51], which relate to the deterministic equivalent of Bayesian networks, and the analysis of game-theoretic equilibria [52].”
BackgroundMethodInfluentialA Socio-technical Approach to Preventing, Mitigating, and Recovering from Ransomware Attacks
Singh, H., & Sittig, D. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2016-04-soa-0064 · Cited by 109 · cited 1×
“Our framework was constructed based on earlier cybersecurity frameworks focusing on SMEs [10,15,92] or STS [5,91,93–95].”
MethodAn open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., Bruin, J. D., Schram, R. D., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B. M. R., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Utrecht University Repository (Utrecht University) · Cited by 106
Systematic Approach to Cyber Resilience Operationalization in SMEs
Carías, J. F., Borges, M. R. S., Labaka, L., Arrizabalaga, S., & Hernantes, J. (2020) · IEEE Access · doi:10.1109/access.2020.3026063 · Cited by 101 · cited 10×
“Manage asset changes and periodicallymaintainassets[92].”
BackgroundMethodInfluentialEngineering Quality and Reliability in Technology-Assisted Review
Cormack, G. V., & Grossman, M. R. (2016) · Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval · doi:10.1145/2911451.2911510 · Cited by 98 · cited 1×
“We stopped evaluating research when the last 20 reviewed papers were considered irrelevant, although more sophisticated stopping criteria exist that are worth considering [72].”
MethodCyber Defense and Situational Awareness
Kott, A., Wang, C., & Erbacher, R. F. (2014) · Advances in information security · doi:10.1007/978-3-319-11391-3 · Cited by 95
STRAM: Measuring the Trustworthiness of Computer-Based Systems
Cho, J.-H., Xu, S., Hurley, P. M., Mackay, M., Benjamin, T., & Beaumont, M. (2019) · ACM Computing Surveys · doi:10.1145/3277666 · Cited by 94 · cited 10×
“Our final research question covered the consideration of adaptability: “the state of being able to change to work or fit better” [18].”
BackgroundMethodInfluentialSecurity Events and Vulnerability Data for Cybersecurity Risk Estimation
Allodi, L., & Massacci, F. (2017) · Risk Analysis · doi:10.1111/risa.12864 · Cited by 92 · cited 1×
“[68], • Noel and Jajodia [69], • Spruit and Roeling [70], • Allodi and Massacci [71], • Cho et al.”
BackgroundToward the automation of threat modeling and risk assessment in IoT systems
Casola, V., De Benedictis, A., Rak, M., & Villano, U. (2019) · Internet of Things · doi:10.1016/j.iot.2019.100056 · Cited by 92 · cited 1×
“Threat-based approaches are ideally suited to this requirement, as general incident data are widely available [99], and can be mapped to threats to offer SMEs insight into what is important for them [100].”
BackgroundRefsdal, A., Solhaug, B., & Stlen, K. (2015) · SpringerBriefs in computer science · doi:10.1007/978-3-319-23570-7 · Cited by 82
SYMBALS: A Systematic Review Methodology Blending Active Learning and Snowballing
van Haastrecht, M., Sarhan, I., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.685591 · Cited by 80 · cited 2×
“…broad coverage of the cybersecurity metrics field, we employed a novel systematic review methodology blending active learning and snowballing (SYMBALS, [58]), which combines existing methods into a swift and accessible methodology, while following authoritative systematic review guidelines [59–61].”
MethodSelf-citationValidation of a socio-technical management process for optimising cybersecurity practices
Malatji, M., Marnewick, A. L., & von Solms, S. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101846 · Cited by 79 · cited 4×
“Embed and automate cy-bersecurity processes [5,10,15,91], which, combined with collaborative stakeholder relationships [92], promote internal and external trust in the SME cyber-security posture [94].”
BackgroundMethodInfluentialQuality assessment of systematic reviews in software engineering: a tertiary study
Zhou, Y., Zhang, H., Huang, X., Yang, S., Babar, M. A., & Tang, H. (2015) · Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2745802.2745815 · Cited by 77 · cited 1×
“Commonly used research quality criteria were adapted for use with a Likert scale [73].”
MethodOrganizational Characteristics Influencing SME Information Security Maturity
Mijnhardt, F., Baars, T., & Spruit, M. R. (2016) · Journal of Computer Information Systems · doi:10.1080/08874417.2016.1117369 · Cited by 74 · cited 1×
“Although automation is a promising approach, the diverse nature of the SME landscape is often ignored [13,14], whereas we know from earlier research that it is vital for SMEs to have solutions adapted to their context and needs [15,16].”
BackgroundSelf-citationStolfo, S., Bellovin, S. M., & Evans, D. (2011) · IEEE Security & Privacy · doi:10.1109/msp.2011.56 · Cited by 71 · cited 1×
“[68], • Noel and Jajodia [69], • Spruit and Roeling [70], • Allodi and Massacci [71], • Cho et al.”
BackgroundCultivating cybersecurity learning: An integration of self-determination and flow
Kam, H.-J., Menard, P., Ormond, D., & Crossler, R. E. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101875 · Cited by 70 · cited 2×
“Altogether, this ensures optimal organisation and employee motivation [25,97].”
BackgroundGrunske, L., & Joyce, D. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2007.11.716 · Cited by 65 · cited 1×
“…[106] Enterprise (A) AW WLC No Villarrubia et al. [107] Enterprise (A) RE WLC No Bhilare et al. [74] Enterprise (M/L) RE None Yes Grunske and Joyce [108] Other DE, AB WP, WM Yes Sahinoglu [84] Other RE WLC, BN Yes Dantu et al. [85] Other DE, AB BN Yes Chen and Wang [87] Other AW WP No Chan [88]…”
BackgroundMetrics suite for network attack graph analytics
Noel, S., & Jajodia, S. (2014) · doi:10.1145/2602087.2602117 · Cited by 63 · cited 1×
“[68], • Noel and Jajodia [69], • Spruit and Roeling [70], • Allodi and Massacci [71], • Cho et al.”
BackgroundAssessing information security risk for an evolving smart city based on fuzzy and grey FMEA
Li, X., Li, H., Sun, B., & Wang, F. (2018) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-172097 · Cited by 60 · cited 2×
“…KN, AB, RE None No Ramos et al. [41] Other RE WLC No Rass et al. [52] Other RE WLC Yes Alohali et al. [128] Other AW, KN, AB WLC, WM No Li et al. [82] Other AW, RE WP No Morrison et al. [43] Other DE, AB, RE WLC No Pramod and Bharathi [129] Enterprise (A) AW, KN, RE WLC, WM No Proença and…”
BackgroundMethodHow Effective Is Your Security Awareness Program? An Evaluation Methodology
Rantos, K., Fysarakis, K., & Manifavas, C. (2012) · Information Security Journal A Global Perspective · doi:10.1080/19393555.2012.747234 · Cited by 59 · cited 1×
“…et al. [78] Other AW, KN WLC No Bojanc et al. [109] Enterprise (A) RE WLC, WM Yes Lo and Chen [50] Enterprise (M/L) AW, RE WLC No Rantos et al. [110] Enterprise (A) AW, DE, KN, AB, RE WLC Yes Shameli-Sendi et al. [80] Enterprise (A) KN, AB WC No Bojanc and Jerman-Blažiˇc [111] Enterprise (A)…”
Backgroundda Veiga, A. (2018) · Information and Computer Security · doi:10.1108/ics-08-2017-0056 · Cited by 56 · cited 2×
“ADKAR has previously been applied in assessing information security culture within organisations [28].”
MethodMapping the field of software life cycle security metrics
Morrison, P. J., Moye, D., Pandita, R., & Williams, L. A. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.05.011 · Cited by 52 · cited 3×
“…RE WLC No Rass et al. [52] Other RE WLC Yes Alohali et al. [128] Other AW, KN, AB WLC, WM No Li et al. [82] Other AW, RE WP No Morrison et al. [43] Other DE, AB, RE WLC No Pramod and Bharathi [129] Enterprise (A) AW, KN, RE WLC, WM No Proença and Borbinha [89] Enterprise (M/L) RE WM No Rueda…”
BackgroundInfluentialNetwork risk management using attacker profiling
Dantu, R., Kolan, P., & Cangussu, J. W. (2008) · Security and Communication Networks · doi:10.1002/sec.58 · Cited by 50 · cited 1×
“…Bhilare et al. [74] Enterprise (M/L) RE None Yes Grunske and Joyce [108] Other DE, AB WP, WM Yes Sahinoglu [84] Other RE WLC, BN Yes Dantu et al. [85] Other DE, AB BN Yes Chen and Wang [87] Other AW WP No Chan [88] Enterprise (M/L) RE WLC, WP No Shin et al. [78] Other AW, KN WLC No Bojanc et al.…”
BackgroundA Quantitative Model for Information-Security Risk Management
Bojanc, R., & Jerman-Blažič, B. (2013) · Engineering Management Journal · doi:10.1080/10429247.2013.11431972 · Cited by 50 · cited 1×
“…No Rantos et al. [110] Enterprise (A) AW, DE, KN, AB, RE WLC Yes Shameli-Sendi et al. [80] Enterprise (A) KN, AB WC No Bojanc and Jerman-Blažiˇc [111] Enterprise (A) AW, RE WLC Yes Marconato et al. [79] Other AW, DE None No Taubenberger et al. [112] Enterprise (M/L) AW, RE None No Alencar Rigon…”
BackgroundMeasuring team effectiveness in cyber-defense exercises: a cross-disciplinary case study
Granåsen, M., & Andersson, D. (2015) · Cognition Technology & Work · doi:10.1007/s10111-015-0350-2 · Cited by 47 · cited 1×
“…Other AW, RE WLC No Brožová et al. [51] Enterprise (A) AW, KN, RE WLC No Brynielsson et al. [124] Other AW, KN, AB None No Granåsen and Andersson [125] Other AW, DE, AB, RE WLC No Orojloo and Azgomi [126] Other KN, AB WLC, WM, WP Yes Aiba and Hiromatsu [127] Enterprise (A) RE WLC Yes Damenu and…”
BackgroundModelling adaptive information security for SMEs in a cluster
Özkan, B. Y., Spruit, M. R., Wondolleck, R., & Burriel, V. (2019) · Journal of Intellectual Capital · doi:10.1108/jic-05-2019-0128 · Cited by 45 · cited 2×
“Although automation is a promising approach, the diverse nature of the SME landscape is often ignored [13,14], whereas we know from earlier research that it is vital for SMEs to have solutions adapted to their context and needs [15,16].”
BackgroundSelf-citationPhysical Intrusion Games—Optimizing Surveillance by Simulation and Game Theory
Raß, S., Alshawish, A., Abid, M. A., Schauer, S., Zhu, Q., & de Meer, H. (2017) · IEEE Access · doi:10.1109/access.2017.2693425 · Cited by 44 · cited 2×
“Two examples of other possibilities are the use of analytic network process (ANP) techniques [50,51], which relate to the deterministic equivalent of Bayesian networks, and the analysis of game-theoretic equilibria [52].”
BackgroundPenQuest: a gamified attacker/defender meta model for cyber security assessment and education
Luh, R., Temper, M., Tjoa, S., Schrittwieser, S., & Janicke, H. (2019) · Journal of Computer Virology and Hacking Techniques · doi:10.1007/s11416-019-00342-x · Cited by 44 · cited 2×
“The former approach was not seen in research, whereas the latter was (for example, [87–90]).”
BackgroundResultŠolić, K., Očevčić, H., & Golub, M. (2015) · Computers & Security · doi:10.1016/j.cose.2015.08.004 · Cited by 42 · cited 1×
“…(M/L) RE WLC, WP Yes Juliadotter and Choo [119] Other AW, DE, KN, AB, RE WLC Yes Otero [120] Enterprise (M/L) AW, AB, RE None Yes Solic et al. [121] Enterprise (A) AW, KN, AB WLC No Sugiura et al. [122] Enterprise (A) AW, RE None No Wei et al. [123] Other AW, AB WLC No You et al. [75] Other…”
BackgroundRisk Assessment for Physical and Cyber Attacks on Critical Infrastructures
DePoy, J., Phelan, J. M., Sholander, P., Smith, B. W., Varnado, G. B., & Wyss, G. D. (2006) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom.2005.1605959 · Cited by 41 · cited 1×
“Dantu and Kolan [83] Other DE, AB BN Yes Depoy et al. [105] Other DE, AB WP, WCP Yes Hasle et al. [106] Enterprise (A) AW WLC No Villarrubia et al. [107] Enterprise (A) RE WLC No Bhilare et al. [74] Enterprise (M/L) RE None Yes Grunske and Joyce [108] Other DE, AB WP, WM Yes Sahinoglu [84] Other RE…”
BackgroundEckhart, M., Brenner, B., Ekelhart, A., & Weippl, E. R. (2019) · Journal of Internet Services and Information Security · Cited by 41
Measuring Risk: Computer Security Metrics, Automation, and Learning
Slayton, R. M. (2015) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2015.30 · Cited by 39 · cited 1×
“Regardless of whether measurement results are deemed relevant by the SME, the knowledge gained by those involved in the measurement process is of value [12].”
BackgroundContinuous Security Metrics for Prevalent Network Threats: Introduction and First Four Metrics
Lippmann, R. P., Riordan, J. F., Yu, T., & Watson, K. K. (2012) · doi:10.21236/ada565825 · Cited by 37 · cited 2×
“The complementary product is another aggregation strategy that is uncommon outside of the security field [49].”
Background(2008) · Springer eBooks · doi:10.1007/978-3-540-68947-8 · Cited by 37
An information security control assessment methodology for organizations' financial information
Otero, A. R. (2015) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2015.06.001 · Cited by 36 · cited 1×
“…None Yes Dehghanimohammadabadi and Bamakan [118] Enterprise (M/L) RE WLC, WP Yes Juliadotter and Choo [119] Other AW, DE, KN, AB, RE WLC Yes Otero [120] Enterprise (M/L) AW, AB, RE None Yes Solic et al. [121] Enterprise (A) AW, KN, AB WLC No Sugiura et al. [122] Enterprise (A) AW, RE None No Wei…”
BackgroundManaging the investment in information security technology by use of a quantitative modeling
Bojanc, R., Jerman-Blažič, B., & Tekavčić, M. (2012) · Information Processing & Management · doi:10.1016/j.ipm.2012.01.001 · Cited by 34 · cited 1×
“…Other DE, AB BN Yes Chen and Wang [87] Other AW WP No Chan [88] Enterprise (M/L) RE WLC, WP No Shin et al. [78] Other AW, KN WLC No Bojanc et al. [109] Enterprise (A) RE WLC, WM Yes Lo and Chen [50] Enterprise (M/L) AW, RE WLC No Rantos et al. [110] Enterprise (A) AW, DE, KN, AB, RE WLC Yes…”
BackgroundTowards a maturity model for health-care cloud security (M2HCS)
Akinsanya, O. O., Papadaki, M., & Sun, L. (2019) · Information and Computer Security · doi:10.1108/ics-05-2019-0060 · Cited by 34 · cited 1×
“…[104] Enterprise (M/L) RE WLC, WM No Stergiopoulos et al. [131] Other AW, KN, AB, RE WLC, WP Yes You et al. [132] Other RE WLC No Akinsanya et al. [133] Other RE WLC No Bharathi [134] Other AW WLC Yes Fertig et al. [135] Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al.…”
BackgroundIOT SECURITY RISK MANAGEMENT MODEL FOR HEALTHCARE INDUSTRY
Salih, F. I., Bakar, N. A. A., Hassan, N. H., Yahya, F., Kama, N., & Shah, J. (2019) · Malaysian Journal of Computer Science · doi:10.22452/mjcs.sp2019no3.9 · Cited by 34 · cited 2×
“[136] Other AW, KN, RE WLC No Cadena et al.”
BackgroundCorporate security solutions for BYOD: A novel user-centric and self-adaptive system
Cuevas, P. D. L., Mora, A. M., Merelo, J. J., Castillo, P. Á., García‐Sánchez, P., & Fernández-Ares, A. J. (2015) · Computer Communications · doi:10.1016/j.comcom.2015.07.019 · Cited by 31 · cited 1×
“It is not surprising to see, then, that adaptability is a key focus of many studies [13,55], although operationalisation of adaptability is still a challenge [53].”
BackgroundCholez, H., & Girard, F. (2013) · Journal of Software Evolution and Process · doi:10.1002/smr.1609 · Cited by 30 · cited 6×
“Successive comparisons of assessment results facilitate continuous process improvement [5,15].”
BackgroundMethodInfluentialŞahinoglu, M. (2008) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2007.915139 · Cited by 28 · cited 1×
“…et al. [107] Enterprise (A) RE WLC No Bhilare et al. [74] Enterprise (M/L) RE None Yes Grunske and Joyce [108] Other DE, AB WP, WM Yes Sahinoglu [84] Other RE WLC, BN Yes Dantu et al. [85] Other DE, AB BN Yes Chen and Wang [87] Other AW WP No Chan [88] Enterprise (M/L) RE WLC, WP No Shin et al.…”
BackgroundAnalysing information security in a bank using soft systems methodology
Damenu, T. K., & Beaumont, C. D. (2017) · Information and Computer Security · doi:10.1108/ics-07-2016-0053 · Cited by 27 · cited 1×
“…AW, DE, AB, RE WLC No Orojloo and Azgomi [126] Other KN, AB WLC, WM, WP Yes Aiba and Hiromatsu [127] Enterprise (A) RE WLC Yes Damenu and Beaumont [103] Enterprise (M/L) AW, DE, KN, AB, RE None No Ramos et al. [41] Other RE WLC No Rass et al. [52] Other RE WLC Yes Alohali et al. [128] Other AW,…”
BackgroundUnknown Vulnerability Risk Assessment Based on Directed Graph Models: A Survey
He, W., Li, H., & Li, J. (2019) · IEEE Access · doi:10.1109/access.2019.2954092 · Cited by 26 · cited 1×
“…× Ramos et al. [41] 2017 Network Security Cherdantseva et al. [42] 2016 SCADA Systems Morrison et al. [43] 2018 Software Security × He et al. [44] 2019 Unknown Vulnerabilities × Xie et al. [45] 2019 Wireless Networks × Some exceptions are comprehensive and cover both social and technical…”
BackgroundISFAM: THE INFORMATION SECURITY FOCUS AREA MATURITY MODEL
Spruit, M. R., & Röling, M. (2014) · Cited by 25
Self-citationA Critical Survey of Security Indicator Approaches
Rudolph, M., & Schwarz, R. (2012) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1109/ares.2012.10 · Cited by 25 · cited 1×
“Rudolph and Schwarz [30] 2012 Generic × Pendleton et al. [17] 2016 Generic Cho et al. [18] 2019 Generic Husák et al. [31] 2019 Attack Prediction Iannacone and Bridges [32] 2020 Cyber Defense × Kordy et al. [33] 2014 Directed Acyclic Graphs × Cadena et al. [34] 2020 Incident Management Knowles et…”
BackgroundCombatting Cybercrime and Cyberterrorism
Akhgar, B., & Brewster, B. (2016) · Advanced sciences and technologies for security applications · doi:10.1007/978-3-319-38930-1 · Cited by 24
Fuzzy Multi-Criteria Decision-Making for Information Security Risk Assessment
Shameli‐Sendi, A. (2012) · The Open Cybernetics & Systemics Journal · doi:10.2174/1874110x01206010026 · Cited by 22 · cited 2×
“…RE WLC, WM Yes Lo and Chen [50] Enterprise (M/L) AW, RE WLC No Rantos et al. [110] Enterprise (A) AW, DE, KN, AB, RE WLC Yes Shameli-Sendi et al. [80] Enterprise (A) KN, AB WC No Bojanc and Jerman-Blažiˇc [111] Enterprise (A) AW, RE WLC Yes Marconato et al. [79] Other AW, DE None No Taubenberger…”
BackgroundMethodBamakan, S. M. H., & Dehghanimohammadabadi, M. (2015) · International Journal of Enterprise Information Systems · doi:10.4018/ijeis.2015100103 · Cited by 22 · cited 1×
“…WP No Suhartana et al. [116] Enterprise (A) AB, RE WLC Yes Yadav and Dong [117] Other AW, KN, AB, RE None Yes Dehghanimohammadabadi and Bamakan [118] Enterprise (M/L) RE WLC, WP Yes Juliadotter and Choo [119] Other AW, DE, KN, AB, RE WLC Yes Otero [120] Enterprise (M/L) AW, AB, RE None Yes…”
BackgroundBharathi, S. V. (2019) · Benchmarking An International Journal · doi:10.1108/bij-08-2018-0264 · Cited by 22 · cited 1×
“…WM No Stergiopoulos et al. [131] Other AW, KN, AB, RE WLC, WP Yes You et al. [132] Other RE WLC No Akinsanya et al. [133] Other RE WLC No Bharathi [134] Other AW WLC Yes Fertig et al. [135] Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al. [136] Other AW, KN, RE WLC No…”
BackgroundCyber Resilience Progression Model
Carías, J. F., Arrizabalaga, S., Labaka, L., & Hernantes, J. (2020) · Applied Sciences · doi:10.3390/app10217393 · Cited by 22 · cited 2×
“Another issue is that cybersecurity assessment approaches aimed at SMEs are still scarce [6], explaining why it is not uncommon to see results from other cybersecurity focus areas being applied to the SME setting [10].”
BackgroundA cyclical evaluation model of information security maturity
Rigon, E. A., Westphall, C. M., dos Santos, D. R., & Westphall, C. B. (2014) · Information Management & Computer Security · doi:10.1108/imcs-04-2013-0025 · Cited by 21 · cited 2×
“…(A) AW, RE WLC Yes Marconato et al. [79] Other AW, DE None No Taubenberger et al. [112] Enterprise (M/L) AW, RE None No Alencar Rigon et al. [102] Enterprise (M/L) RE WLC No Boggs et al. [113] Other AW WLC Yes Chen et al. [114] Enterprise (M/L) RE WLC No Cheng et al. [115] Other AW WLC, WP,…”
BackgroundIannacone, M. D., & Bridges, R. A. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101907 · Cited by 20
Comparison of Adaptive Information Security Approaches
Evesti, A., & Ovaska, E. (2013) · ISRN Artificial Intelligence · doi:10.1155/2013/482949 · Cited by 19 · cited 2×
“Adaptability is crucial to any cybersecurity solution [53].”
BackgroundPredicting the behavior of attackers and the consequences of attacks against cyber‐physical systems
Orojloo, H., & Azgomi, M. A. (2016) · Security and Communication Networks · doi:10.1002/sec.1761 · Cited by 19 · cited 1×
“…(A) AW, KN, RE WLC No Brynielsson et al. [124] Other AW, KN, AB None No Granåsen and Andersson [125] Other AW, DE, AB, RE WLC No Orojloo and Azgomi [126] Other KN, AB WLC, WM, WP Yes Aiba and Hiromatsu [127] Enterprise (A) RE WLC Yes Damenu and Beaumont [103] Enterprise (M/L) AW, DE, KN, AB, RE…”
BackgroundResolving vulnerability identification errors using security requirements on business process models
Taubenberger, S., Jürjens, J., Yu, Y., & Nuseibeh, B. (2013) · Information Management & Computer Security · doi:10.1108/imcs-09-2012-0054 · Cited by 18
Using formal distributions for threat likelihood estimation in cloud-enabled IT risk assessment
Stergiopoulos, G., Gritzalis, D. A., & Kouktzoglou, V. (2018) · Computer Networks · doi:10.1016/j.comnet.2018.01.033 · Cited by 18 · cited 1×
“…WM No Rueda and Avila [130] Enterprise (M/L) AW, KN, AB, RE WLC, WP No Shokouhyar et al. [104] Enterprise (M/L) RE WLC, WM No Stergiopoulos et al. [131] Other AW, KN, AB, RE WLC, WP Yes You et al. [132] Other RE WLC No Akinsanya et al. [133] Other RE WLC No Bharathi [134] Other AW WLC Yes Fertig…”
BackgroundRhee, K.-H., & Yi, J. H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-15087-1 · Cited by 18
Information Systems Security Assessment Based on System Dynamics
Wei, L. I. U., Cui, Y.-F., & Ya, L. (2015) · International Journal of Security and Its Applications · doi:10.14257/ijsia.2015.9.2.08 · Cited by 17 · cited 1×
“…(M/L) AW, AB, RE None Yes Solic et al. [121] Enterprise (A) AW, KN, AB WLC No Sugiura et al. [122] Enterprise (A) AW, RE None No Wei et al. [123] Other AW, AB WLC No You et al. [75] Other AW, RE WLC No Brožová et al. [51] Enterprise (A) AW, KN, RE WLC No Brynielsson et al. [124] Other AW,…”
BackgroundA Comprehensive Method to Assess Work System Security Risk
Yadav, S. B., & Dong, T. (2014) · Communications of the Association for Information Systems · doi:10.17705/1cais.03408 · Cited by 16 · cited 1×
“…Enterprise (A) AW, DE, KN, AB, RE WLC Yes Silva et al. [81] Other RE WLC, WP No Suhartana et al. [116] Enterprise (A) AB, RE WLC Yes Yadav and Dong [117] Other AW, KN, AB, RE None Yes Dehghanimohammadabadi and Bamakan [118] Enterprise (M/L) RE WLC, WP Yes Juliadotter and Choo [119] Other AW, DE,…”
BackgroundJuliadotter, N. V., & Choo, K. R. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-801595-7.00003-3 · Cited by 15 · cited 1×
“…Yes Yadav and Dong [117] Other AW, KN, AB, RE None Yes Dehghanimohammadabadi and Bamakan [118] Enterprise (M/L) RE WLC, WP Yes Juliadotter and Choo [119] Other AW, DE, KN, AB, RE WLC Yes Otero [120] Enterprise (M/L) AW, AB, RE None Yes Solic et al. [121] Enterprise (A) AW, KN, AB WLC No Sugiura et…”
BackgroundA Vulnerability Life Cycle-Based Security Modeling and Evaluation Approach
Marconato, G., Kaâniche, M., & Nicomette, V. (2012) · The Computer Journal · doi:10.1093/comjnl/bxs112 · Cited by 15 · cited 2×
“In papers covering the defender, it is quite common to ignore threats entirely [78] or to use a proxy such as the prevalence of vulnerabilities to represent threats [79].”
BackgroundAn analytics approach to adaptive maturity models using organizational characteristics
Baars, T., Mijnhardt, F., Vlaanderen, K., & Spruit, M. R. (2016) · Decision Analytics · doi:10.1186/s40165-016-0022-1 · Cited by 15 · cited 1×
“Especially when measuring cybersecurity, a rigid solution that does not adapt to a changing environment or a new use case is far from optimal [54].”
BackgroundSelf-citationThreat-Based Risk Assessment for Enterprise Networks
Riordan, J. F., & Lippmann, R. P. (2016) · Cited by 13
DSAPE – Dynamic Security Awareness Program Evaluation
Manifavas, C., Fysarakis, K., Rantos, K., & Hatzivasilis, G. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07620-1_23 · Cited by 12 · cited 2×
“…al. [114] Enterprise (M/L) RE WLC No Cheng et al. [115] Other AW WLC, WP, WM Yes Feng et al. [86] Enterprise (M/L) AW, RE BN Yes Manifavas et al. [77] Enterprise (A) AW, DE, KN, AB, RE WLC Yes Silva et al. [81] Other RE WLC, WP No Suhartana et al. [116] Enterprise (A) AB, RE WLC Yes Yadav and…”
BackgroundAutomating the Communication of Cybersecurity Knowledge: Multi-case Study
Shojaifar, A., Fricker, S. A., & Gwerder, M. (2020) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-59291-2_8 · Cited by 12 · cited 5×
“A solution to this issue is to automate the cybersecurity assessment process where possible [9].”
BackgroundMethodInfluentialIntelligence and Security Informatics
Chen, H., Zeng, D. D., Wang, F.-Y., Roberts, F. S., Mureşan, G., Merkle, R. C., & Kantor, P. (2005) · Lecture notes in computer science · doi:10.1007/b136511 · Cited by 11
You, Y., Oh, J., Kim, S., & Lee, K. (2018) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2018.10.020 · Cited by 11 · cited 1×
“…KN, AB, RE WLC, WP No Shokouhyar et al. [104] Enterprise (M/L) RE WLC, WM No Stergiopoulos et al. [131] Other AW, KN, AB, RE WLC, WP Yes You et al. [132] Other RE WLC No Akinsanya et al. [133] Other RE WLC No Bharathi [134] Other AW WLC Yes Fertig et al. [135] Other AW, DE, KN None No Husák et al.…”
BackgroundMeasuring Drive-by Download Defense in Depth
Boggs, N., Du, S., & Stolfo, S. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11379-1_9 · Cited by 10 · cited 1×
“…Other AW, DE None No Taubenberger et al. [112] Enterprise (M/L) AW, RE None No Alencar Rigon et al. [102] Enterprise (M/L) RE WLC No Boggs et al. [113] Other AW WLC Yes Chen et al. [114] Enterprise (M/L) RE WLC No Cheng et al. [115] Other AW WLC, WP, WM Yes Feng et al. [86] Enterprise (M/L) AW,…”
BackgroundLearning and Collaboration Technologies. Designing Learning Experiences
(2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21814-0 · Cited by 10
Ghabri, R., Hirmer, P., & Mitschang, B. (2018) · Lecture notes in business information processing · doi:10.1007/978-3-319-93931-5 · Cited by 10
Metrics of Password Management Policy
Villarrubia, C., Fernández‐Medina, E., & Piattini, M. G. (2006) · Lecture notes in computer science · doi:10.1007/11751595_106 · Cited by 9 · cited 1×
“Dantu and Kolan [83] Other DE, AB BN Yes Depoy et al. [105] Other DE, AB WP, WCP Yes Hasle et al. [106] Enterprise (A) AW WLC No Villarrubia et al. [107] Enterprise (A) RE WLC No Bhilare et al. [74] Enterprise (M/L) RE None Yes Grunske and Joyce [108] Other DE, AB WP, WM Yes Sahinoglu [84] Other RE…”
BackgroundInformation Security Practice and Experience
Bao, F., Zhou, J., Pang, H., & Deng, R. H. (2005) · Lecture notes in computer science · doi:10.1007/b107167 · Cited by 9
Alohali, M. A., Clarke, N., & Furnell, S. (2018) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2018.091018 · Cited by 9 · cited 1×
“…and Beaumont [103] Enterprise (M/L) AW, DE, KN, AB, RE None No Ramos et al. [41] Other RE WLC No Rass et al. [52] Other RE WLC Yes Alohali et al. [128] Other AW, KN, AB WLC, WM No Li et al. [82] Other AW, RE WP No Morrison et al. [43] Other DE, AB, RE WLC No Pramod and Bharathi [129] Enterprise…”
BackgroundAddressing SME Characteristics for Designing Information Security Maturity Models
Özkan, B. Y., & Spruit, M. R. (2020) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-57404-8_13 · Cited by 9 · cited 1×
“To address the singular characteristics of our setting, we additionally incorporated the findings from our systematic review, as well as principles for designing cybersecurity maturity models for SMEs [96], in our framework.”
MethodSelf-citationChen, M., & Wang, S. (2010) · Kybernetes · doi:10.1108/03684921011043260 · Cited by 8 · cited 2×
“The former approach was not seen in research, whereas the latter was (for example, [87–90]).”
BackgroundResultMetrics and Indicators of Information Security Incident Management: A Systematic Mapping Study
Cadena, A., Gualoto, F., Fuertes, W., Tello-Oquendo, L., Andrade, R., Tapia, F., & Torres, J. (2019) · Smart innovation, systems and technologies · doi:10.1007/978-981-13-9155-2_40 · Cited by 7 · cited 2×
“…AW WLC Yes Fertig et al. [135] Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al. [136] Other AW, KN, RE WLC No Cadena et al. [34] Other AW None No Wirtz and Heisel [137] Other AW, KN, AB, RE WLC, WM No Ganin et al. [138] Enterprise (A) AW, DE, KN, AB, RE WLC No Luh et al.…”
BackgroundMeasuring the Impact of E-Learning Platforms on Information Security Awareness
Fertig, T., Schütz, A. E., Weber, K., & Müller, N. H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21814-0_3 · Cited by 7 · cited 1×
“…AW, KN, AB, RE WLC, WP Yes You et al. [132] Other RE WLC No Akinsanya et al. [133] Other RE WLC No Bharathi [134] Other AW WLC Yes Fertig et al. [135] Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al. [136] Other AW, KN, RE WLC No Cadena et al. [34] Other AW None No Wirtz…”
BackgroundInformation Security Risk Modeling Using Bayesian Index
Chan, C. (2010) · The Computer Journal · doi:10.1093/comjnl/bxq059 · Cited by 6 · cited 1×
“…[108] Other DE, AB WP, WM Yes Sahinoglu [84] Other RE WLC, BN Yes Dantu et al. [85] Other DE, AB BN Yes Chen and Wang [87] Other AW WP No Chan [88] Enterprise (M/L) RE WLC, WP No Shin et al. [78] Other AW, KN WLC No Bojanc et al. [109] Enterprise (A) RE WLC, WM Yes Lo and Chen [50] Enterprise…”
BackgroundA new information security risk analysis method based on membership degree
Chen, J., Pedrycz, W., Ma, L., & Wang, C. (2014) · Kybernetes · doi:10.1108/k-10-2013-0235 · Cited by 6 · cited 1×
“…et al. [112] Enterprise (M/L) AW, RE None No Alencar Rigon et al. [102] Enterprise (M/L) RE WLC No Boggs et al. [113] Other AW WLC Yes Chen et al. [114] Enterprise (M/L) RE WLC No Cheng et al. [115] Other AW WLC, WP, WM Yes Feng et al. [86] Enterprise (M/L) AW, RE BN Yes Manifavas et al. [77]…”
BackgroundModeling of Risk Factors in Determining Network Security Level
Suhartana, M., Pardamean, B., & Soewito, B. (2014) · International Journal of Security and Its Applications · doi:10.14257/ijsia.2014.8.3.21 · Cited by 6 · cited 1×
“…(M/L) AW, RE BN Yes Manifavas et al. [77] Enterprise (A) AW, DE, KN, AB, RE WLC Yes Silva et al. [81] Other RE WLC, WP No Suhartana et al. [116] Enterprise (A) AB, RE WLC Yes Yadav and Dong [117] Other AW, KN, AB, RE None Yes Dehghanimohammadabadi and Bamakan [118] Enterprise (M/L) RE…”
Background(2018) · Communications in computer and information science · doi:10.1007/978-3-030-01535-0 · Cited by 6
Yang, N., Singh, T., & Johnston, A. (2020) · Journal of the Association for Information Systems · Cited by 6
Socio-Technical Security Metrics (Dagstuhl Seminar 14491)
Gollmann, D., Herley, C., Koenig, V., Pieters, W., & Sasse, M. A. (2015) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dagrep.4.12.1 · Cited by 5
Information Security Management: ANP Based Approach for Risk Analysis and Decision Making
DSE, F., Brožová, H., Šup, L., Rydval, J., Sadok, M., & Bednár, P. (2016) · Agris on-line Papers in Economics and Informatics · doi:10.7160/aol.2016.080102 · Cited by 5 · cited 7×
“Two examples of other possibilities are the use of analytic network process (ANP) techniques [50,51], which relate to the deterministic equivalent of Bayesian networks, and the analysis of game-theoretic equilibria [52].”
BackgroundMethodInfluentialDeveloping an Information Security Risk Taxonomy and an Assessment Model using Fuzzy Petri Nets
Pramod, D., & Bharathi, S. V. (2018) · Journal of Cases on Information Technology · doi:10.4018/jcit.2018070104 · Cited by 5 · cited 1×
“…Alohali et al. [128] Other AW, KN, AB WLC, WM No Li et al. [82] Other AW, RE WP No Morrison et al. [43] Other DE, AB, RE WLC No Pramod and Bharathi [129] Enterprise (A) AW, KN, RE WLC, WM No Proença and Borbinha [89] Enterprise (M/L) RE WM No Rueda and Avila [130] Enterprise (M/L) AW, KN, AB, RE…”
BackgroundModel-Based Risk Analysis and Evaluation Using CORAS and CVSS
Wirtz, R., & Heisel, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-40223-5_6 · Cited by 5 · cited 1×
“…Other AW, DE, KN None No Husák et al. [31] Other AB WLC Yes Salih et al. [136] Other AW, KN, RE WLC No Cadena et al. [34] Other AW None No Wirtz and Heisel [137] Other AW, KN, AB, RE WLC, WM No Ganin et al. [138] Enterprise (A) AW, DE, KN, AB, RE WLC No Luh et al. [90] Other AW, DE, AB, RE WLC Yes”
BackgroundYang, N., Singh, T., & Johnston, A. C. (2020) · Journal of the Association for Information Systems · doi:10.17705/1atrr.00053 · Cited by 4
An information system risk assessment model: a case study in online banking system
Shokouhyar, S., Panahifar, F., Karimisefat, A., & Nezafatbakhsh, M. (2018) · International Journal of Electronic Security and Digital Forensics · doi:10.1504/ijesdf.2018.089205 · Cited by 3
Information Security Assessment and Reporting: Distributed Defense
Bhilare, D. S., Ramani, A. K., & Tanwani, S. (2008) · Cited by 2
Sugiura, M., Suwa, H., & Ohta, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20901-2_8 · Cited by 2 · cited 1×
“…AW, DE, KN, AB, RE WLC Yes Otero [120] Enterprise (M/L) AW, AB, RE None Yes Solic et al. [121] Enterprise (A) AW, KN, AB WLC No Sugiura et al. [122] Enterprise (A) AW, RE None No Wei et al. [123] Other AW, AB WLC No You et al. [75] Other AW, RE WLC No Brožová et al. [51] Enterprise (A) AW,…”
BackgroundAutomating Information Security Risk Assessment for IT Services
Rueda, S. J., & Avila, O. (2018) · Communications in computer and information science · doi:10.1007/978-3-030-01535-0_14 · Cited by 2 · cited 1×
“…AB, RE WLC No Pramod and Bharathi [129] Enterprise (A) AW, KN, RE WLC, WM No Proença and Borbinha [89] Enterprise (M/L) RE WM No Rueda and Avila [130] Enterprise (M/L) AW, KN, AB, RE WLC, WP No Shokouhyar et al. [104] Enterprise (M/L) RE WLC, WM No Stergiopoulos et al. [131] Other AW, KN, AB,…”
BackgroundSocio-technical security metrics
Gollmann, D. (., Herley, C. (., Koenig, V. (., Pieters, W., & Sasse, M. A. (. (2015) · Research Repository (Delft University of Technology) · Cited by 2
Aiba, R., & Hiromatsu, T. (2017) · Journal of Disaster Research · doi:10.20965/jdr.2017.p1060 · Cited by 1 · cited 1×
“…KN, AB None No Granåsen and Andersson [125] Other AW, DE, AB, RE WLC No Orojloo and Azgomi [126] Other KN, AB WLC, WM, WP Yes Aiba and Hiromatsu [127] Enterprise (A) RE WLC Yes Damenu and Beaumont [103] Enterprise (M/L) AW, DE, KN, AB, RE None No Ramos et al. [41] Other RE WLC No Rass et al.…”
Background
Research works printed but not linked: 3
- Jaquith, A. (2007). Security Metrics: Replacing Fear, Uncertainty, and Doubt, Pearson Education.
- Ferguson, N., and Schneier, B. (2003). Practical Cryptography, Wiley.
- Liu, Y., Sarabi, A., Zhang, J., Naghizadeh, P., Karir, M., Bailey, M., and Liu, M. (2015, January 12–14). Cloudy with a Chance of Breach: Forecasting Cyber Security Incidents. Proceedings of the 24th {USENIX} Security Symposium ({USENIX} Security 15), Washington, DC, USA.
Cited by
Ferdinands, G., Schram, R., de Bruin, J., Bagheri, A., Oberski, D. L., Tummers, L., Teijema, J. J., & van de Schoot, R. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02257-7 · Cited by 96
Machine Learning Cybersecurity Adoption in Small and Medium Enterprises in Developed Countries
Rawindaran, N., Jayal, A., & Prakash, E. (2021) · Computers · doi:10.3390/computers10110150 · Cited by 108
Cybersecurity in local governments: A systematic review and framework of key challenges
Hossain, S. T., Yiğitcanlar, T., Nguyen, K., & Xu, Y. (2025) · Urban Governance · doi:10.1016/j.ugj.2024.12.010 · Cited by 65
A Shared Cyber Threat Intelligence Solution for SMEs
van Haastrecht, M., Golpur, G., Tzismadia, G., Kab, R., Priboi, C., David, D., Răcătăian, A., Baumgartner, L., Fricker, S., Ruiz, J., et al. (2021) · Electronics · doi:10.3390/electronics10232913 · Cited by 56
Self-citationNeuro-Symbolic AI and Behavioral Intelligence: Towards Explainable Cybersecurity Mechanisms
Shreha, S. A., Ravikumar, R. N., & Aarthi, S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5012-7.ch005 · Cited by 17
Metadata
- Authors
- van Haastrecht, M., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M.
- Affiliations
- Department of Information and Computing Sciences, Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands
- Utrecht University (NL)
- Leiden University (NL)
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-07-27
- Volume
- 11
- Issue
- 15
- Pages
- 6909
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3184791374
- Cited by
- 45
- References
- 136
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 10 · 2023: 4 · 2022: 3 · 2021: 3
- FWCI
- 0.76
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- Leiden Repository (Leiden University) (publishedVersion, cc-by)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- Horizon 2020 Framework Programme (883588)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
- Social Sciences › Business, Management and Accounting › Management Information Systems › Information Technology Governance and Strategy
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Keywords
- cybersecurity metrics, small and medium-sized enterprises, socio-technical systems, cybersecurity risk assessment, metric aggregation, cybersecurity posture, cybersecurity framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.