Designing Monitoring Systems for Continuous Certification of Cloud Services: Deriving Meta-requirements and Design Guidelines
Abstract
Continuous service certification (CSC) involves the consistently gathering and assessing certification-relevant information about cloud service operations to validate whether they continue to adhere to certification criteria. Previous research has proposed test-based CSC methodologies that directly assess the components of cloud service infrastructures. However, test-based certification requires that certification authorities can access the cloud infrastructure, which various issues may limit. To address these challenges, cloud service providers need to conduct monitoring-based CSC; that is, monitor their cloud service infrastructure to gather certification-relevant data by themselves and then provide these data to certification authorities. Nevertheless, we need to better understand how to design monitoring systems to enable cloud service providers to perform such monitoring. By taking a design science perspective, we derive universal meta-requirements and design guidelines for CSC monitoring systems based on findings from five expert focus group interviews with 33 cloud experts and 10 one-to-one interviews with cloud customers. With this study, we expand the current knowledge base regarding CSC and monitoring-based CSC. Our derived design guidelines contribute to the development of CSC monitoring systems and enable monitoring-based CSC that overcomes issues of prior test-based approaches.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Effective auditing of AI warrants creating an independent, tamper-resistant audit trail and continuous updating and assessment using continuous auditing procedures.
“Effective auditing of AI, therefore, does not only warrant creating an independent, tamper-resistant audit trail, but also the continuous updating and assessment of this audit trail using continuous auditing procedures (Lins et al. 2019).”
From Trustworthy artificial intelligence (2021) · cites this paper as Lins et al. 2019
Cites
Design science in information systems research
(2004) · Journal of the Association for Information Systems · doi:10.5555/2017212.2017217 · Cited by 8,254 · see citing works
Interpretive case studies in IS research: nature and method
Walsham, G. (1995) · European Journal of Information Systems · doi:10.1057/ejis.1995.9 · Cited by 3,834 · see citing works
The SAGE Handbook of Grounded Theory
Bryant, A., & Charmaz, K. (2007) · SAGE · doi:10.4135/9781848607941 · Cited by 2,798 · see citing works
A survey on security issues in service delivery models of cloud computing
Subashini, S., & Kavitha, V. (2010) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2010.07.006 · Cited by 2,692 · see citing works
The qualitative interview in IS research: Examining the craft
Myers, M. D., & Newman, M. (2007) · Information and Organization · doi:10.1016/j.infoandorg.2006.11.001 · Cited by 1,941 · see citing works
Building an Information System Design Theory for Vigilant EIS
Walls, J., Widmeyer, G. R., & Sawy, O. A. E. (1992) · Information Systems Research · doi:10.1287/isre.3.1.36 · Cited by 1,591 · see citing works
Generalizing Generalizability in Information Systems Research
Lee, A. S., & Baskerville, R. (2003) · Information Systems Research · doi:10.1287/isre.14.3.221.16560 · Cited by 1,546 · see citing works
The Anatomy of a Design Theory
Jones, D. J., & Gregor, S. D. (2007) · Journal of the Association for Information Systems · doi:10.17705/1jais.00129 · Cited by 1,518 · see citing works
Levina, N., & Vaast, E. (2005) · MIS Quarterly · doi:10.2307/25148682 · Cited by 1,281 · see citing works
A design theory for systems that support emergent knowledge processes
Markus, M. L., Majchrzak, A., & Gasser, L. (2002) · doi:10.5555/2017167.2017170 · Cited by 1,066 · see citing works
An Efficient and Secure Dynamic Auditing Protocol for Data Storage in Cloud Computing
Yang, K., & Jia, X. (2012) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2012.278 · Cited by 635 · see citing works
On theory development in design science research: anatomy of a research project
Kuechler, B., & Vaishnavi, V. (2008) · European Journal of Information Systems · doi:10.1057/ejis.2008.40 · Cited by 632 · see citing works
Practical data-oriented microaggregation for statistical disclosure control
Domingo-Ferrer, J., & Mateo‐Sanz, J. M. (2002) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.979982 · Cited by 579 · see citing works
Developing multi‐agent systems with a FIPA‐compliant agent framework
Bellifemine, F. L., Poggi, A., & Rimassa, G. (2001) · Software Practice and Experience · doi:10.1002/1097-024x(200102)31:2 · Cited by 577 · see citing works
Inter‐Cloud architectures and application brokering: taxonomy and survey
Grozev, N., & Buyya, R. (2012) · Software Practice and Experience · doi:10.1002/spe.2168 · Cited by 499 · see citing works
Security issues in cloud environments: a survey
Fernandes, D. A. B., Soares, L. F. B., Gomes, J. V., Freire, M. M., & Inácio, P. R. M. (2013) · International Journal of Information Security · doi:10.1007/s10207-013-0208-7 · Cited by 499 · see citing works
Cloud Computing – A Classification, Business Models, and Research Directions
Weinhardt, C., Anandasivam, A., Blau, B., Borissov, N., Meinl, T., Michalk, W., & Stößer, J. (2009) · Business & Information Systems Engineering · doi:10.1007/s12599-009-0071-2 · Cited by 454 · see citing works
Technological Embeddedness and Organizational Change
Volkoff, O., Strong, D. M., & Elmes, M. B. (2007) · Organization Science · doi:10.1287/orsc.1070.0288 · Cited by 447 · see citing works
How does IT ambidexterity impact organizational agility?
Lee, O.-K. (., Sambamurthy, V., Lim, K. H., & Wei, K. K. (2015) · Information Systems Research · doi:10.1287/isre.2015.0577 · Cited by 447 · see citing works
strong, & Volkoff, O. (2010) · MIS Quarterly · doi:10.2307/25750703 · Cited by 423 · see citing works
Service-Oriented Computing and Cloud Computing: Challenges and Opportunities
Wei, Y., & Blake, M. B. (2010) · IEEE Internet Computing · doi:10.1109/mic.2010.147 · Cited by 403 · see citing works
Projective techniques in consumer research
Donoghue, S. (2010) · Journal of Family Ecology and Consumer Sciences / Tydskrif vir Gesinsekologie en Verbruikerswetenskappe · doi:10.4314/jfecs.v28i1.52784 · Cited by 359 · see citing works
Dynamic Audit Services for Outsourced Storages in Clouds
Zhu, Y., Ahn, G., Hu, H., et al. (2011) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2011.51 · Cited by 321 · see citing works
Towards Achieving Data Security with the Cloud Computing Adoption Framework
Chang, V. I. C., & Ramachandran, M. (2015) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2015.2491281 · Cited by 291 · see citing works
Continuous Auditing: Building Automated Auditing Capability
Rezaee, Z., Sharbatoghlie, A., Elam, R., et al. (2002) · Auditing A Journal of Practice & Theory · doi:10.2308/aud.2002.21.1.147 · Cited by 267 · see citing works
Schneider, S., & Sunyaev, A. (2014) · Journal of Information Technology · doi:10.1057/jit.2014.25 · Cited by 257 · see citing works
Self-citationPries-Heje, & Baskerville, R. (2008) · MIS Quarterly · doi:10.2307/25148870 · Cited by 241 · see citing works
Emeakaroha, V. C., Brandić, I., Maurer, M., et al. (2010) · International Conference on High Performance Computing & Simulation (HPCS) · doi:10.1109/hpcs.2010.5547150 · Cited by 220 · see citing works
A combined approach to ensure data security in cloud computing
Sood, S. K. (2012) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2012.07.007 · Cited by 220 · see citing works
GIST: A Model for Design and Management of Content and Interactivity of Customer-Centric Web Sites1
Albert, A., Goes, P. B., & Gupta, A. K. (2004) · MIS Quarterly · doi:10.2307/25148632 · Cited by 171 · see citing works
A survey of Cloud monitoring tools: Taxonomy, capabilities and objectives
Fatema, K., Emeakaroha, V. C., Healy, P. D., Morrison, J. P., & Lynn, T. (2014) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2014.06.007 · Cited by 170 · see citing works
Understanding enterprise systems-enabled integration
Volkoff, O., Strong, D. M., & Elmes, M. B. (2005) · European Journal of Information Systems · doi:10.1057/palgrave.ejis.3000528 · Cited by 147 · see citing works
A Theory-Driven Design Framework for Social Recommender Systems
Arazy, O., Kumar, N. S., & Shapira, B. (2010) · Journal of the Association for Information Systems · doi:10.17705/1jais.00237 · Cited by 135 · see citing works
Sunyaev, A., & Schneider, S. (2013) · Communications of the ACM · doi:10.1145/2408776.2408789 · Cited by 131 · see citing works
Self-citationCloud forensics: Technical challenges, solutions and comparative analysis
Pichan, A., Lazarescu, M., & Soh, S. (2015) · Digital Investigation · doi:10.1016/j.diin.2015.03.002 · Cited by 130 · see citing works
DARGOS: A highly adaptable and scalable monitoring architecture for multi-tenant Clouds
Povedano-Molina, J., Lopez-Vega, J. M., Lopez-Soler, J. M., et al. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2013.04.022 · Cited by 128 · see citing works
Trust is Good, Control is Better: Creating Secure Clouds by Continuous Auditing
Lins, S., Schneider, S., & Sunyaev, A. (2016) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2016.2522411 · Cited by 120 · see citing works
Self-citationUnderstanding IS evaluation as a complex social process: a case study of a UK local authority
Jones, S. R., & Hughes, J. (2001) · European Journal of Information Systems · doi:10.1057/palgrave.ejis.3000405 · Cited by 116 · see citing works
From the Service-Oriented Architecture to the Web API Economy
Tan, W., Fan, Y., Ghoneim, A., Hossain, M. A., & Dustdar, S. (2016) · IEEE Internet Computing · doi:10.1109/mic.2016.74 · Cited by 111 · see citing works
Calero, J. M. A., & Aguado, J. (2014) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2014.2302810 · Cited by 108 · see citing works
The Grounded Theory Method and Case Study Data in IS Research: Issues and Design
Fernandez, W. D. (2005) · ANU Open Research (Australian National University) · Cited by 107 · see citing works
Massonet, P., Naqvi, S. M. A. A., Ponsard, C., et al. (2011) · doi:10.1109/ipdps.2011.304 · Cited by 87 · see citing works
Theory-generating design science research
Beck, R., Weber, S., & Gregory, R. W. (2012) · Information Systems Frontiers · doi:10.1007/s10796-012-9342-4 · Cited by 72 · see citing works
Accountability for Data Governance in Cloud Ecosystems
Felici, M. D., Koulouris, T., & Pearson, S. (2013) · IEEE International Conference on Cloud Computing Technology and Science · doi:10.1109/cloudcom.2013.157 · Cited by 67 · see citing works
An agent based business aware incident detection system for cloud environments
Doelitzscher, F., Reich, C., Knahl, M., et al. (2012) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/2192-113x-1-9 · Cited by 67 · see citing works
Exploring the influences and use of the literature during a grounded theory study
Heath, H. (2006) · Journal of research in nursing · doi:10.1177/1744987106069338 · Cited by 66 · see citing works
Cloud monitoring: A review, taxonomy, and open research issues
Syed, H. J., Gani, A., Ahmad, R. W., Khan, M. K., & Ahmed, A. I. A. (2017) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.08.021 · Cited by 64 · see citing works
Buyya, R., Ranjan, R. K., & Calheiros, R. N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1003.3920 · Cited by 60 · see citing works
Using a grounded theory approach to study online collaboration behaviors
Gasson, S., & Waters, J. (2011) · European Journal of Information Systems · doi:10.1057/ejis.2011.24 · Cited by 59 · see citing works
Automated Certification for Compliant Cloud-based Business Processes
Accorsi, R., Lowis, L., & Sato, Y. (2011) · Business & Information Systems Engineering · doi:10.1007/s12599-011-0155-7 · Cited by 59 · see citing works
A Semi-Automatic and Trustworthy Scheme for Continuous Cloud Service Certification
Anisetti, M., Ardagna, C. A., Damiani, E., et al. (2017) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2017.2657505 · Cited by 58 · see citing works
Dynamic Certification of Cloud Services: Trust, but Verify!
Lins, S., Grochol, P., Schneider, S., et al. (2016) · IEEE Security & Privacy · doi:10.1109/msp.2016.26 · Cited by 56 · see citing works
Self-citationContinuous auditing with a multi-agent system
Chou, C. L.-Y., DU, T. C.-T., & Lai, V. S. K. (2006) · Decision Support Systems · doi:10.1016/j.dss.2006.08.002 · Cited by 52 · see citing works
Current cloud challenges in Germany: the perspective of cloud service providers
Hentschel, R., Leyh, C., & Petznick, A. (2018) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-018-0107-6 · Cited by 52 · see citing works
Gao, F., Thiebes, S., & Sunyaev, A. (2018) · Journal of Medical Internet Research · doi:10.2196/10041 · Cited by 52 · see citing works
Software architectures: Blueprint, Literature, Language or Decision?
Smolander, K., Rossi, M., & Purao, S. (2008) · European Journal of Information Systems · doi:10.1057/ejis.2008.48 · Cited by 49 · see citing works
Enriching our theoretical repertoire: the role of evolutionary psychology in technology acceptance
Abraham, C., Boudreau, M., Junglas, I. A., & Watson, R. T. (2011) · European Journal of Information Systems · doi:10.1057/ejis.2011.25 · Cited by 45 · see citing works
Building a Service-Oriented Monitoring Framework with REST and Nagios
Katsaros, G., Kübert, R., & Gallizo, G. (2011) · IEEE International Conference on Services Computing · doi:10.1109/scc.2011.53 · Cited by 44 · see citing works
Schneider, S., Wollersheim, J., Krcmar, H., & Sunyaev, A. (2016) · Journal of Information Technology · doi:10.1057/s41265-016-0001-y · Cited by 43 · see citing works
Self-citationValidating Cloud Infrastructure Changes by Cloud Audits
Doelitzscher, F., Fischer, C., Moskal, D., et al. (2012) · IEEE World Congress on Services (SERVICES) · doi:10.1109/services.2012.12 · Cited by 39 · see citing works
Schneider, S., Lansing, J., Gao, F., et al. (2014) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2014.614 · Cited by 38 · see citing works
Self-citationLansing, J., Benlian, A., Darmstadt, T. U., & Sunyaev, A. (2018) · Journal of the Association for Information Systems · doi:10.17705/1jais.00520 · Cited by 36 · see citing works
Policies and Research in Identity Management
Fischer‐Hübner, S., de Leeuw, E., & Mitchell, C. (2013) · IFIP International Federation for Information Processing/IFIP · doi:10.1007/978-3-642-37282-7 · Cited by 31 · see citing works
VMDriver: A Driver-Based Monitoring Mechanism for Virtualization
Xiang, G., Jin, H., Zou, D., et al. (2010) · Proceedings/Proceedings - Symposium on Reliable Distributed Systems · doi:10.1109/srds.2010.38 · Cited by 30 · see citing works
Trust in Cloud Services: Providing More Controls to Clients
Khan, K. M., & Malluhi, Q. M. (2013) · Computer · doi:10.1109/mc.2013.254 · Cited by 28 · see citing works
Test-Based Cloud Service Certification of Opportunistic Providers
Stephanow, P., Srivastava, G., & Schütte, J. (2016) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2016.0120 · Cited by 23 · see citing works
A few new approaches for data masking
Sarada, G., Abitha, N., Manikandan, G., et al. (2015) · International Conference on Circuits, Power and Computing Technologies (ICCPCT) · doi:10.1109/iccpct.2015.7159301 · Cited by 22 · see citing works
Design of Data Masking Architecture and Analysis of Data Masking Techniques for Testing
Ravikumar, G. K., Rabi, B. J., Manjunath, T. N., et al. (2011) · Cited by 20 · see citing works
Certification-Based Cloud Adaptation
Ardagna, C. A., Asal, R., Damiani, E., et al. (2018) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2018.2793268 · Cited by 20 · see citing works
Dynamic Certification of Cloud Services
Windhorst, I., & Sunyaev, A. (2013) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1109/ares.2013.55 · Cited by 19 · see citing works
Dynamic set-up of monitoring infrastructures for service based systems
Comuzzi, M., & Spanoudakis, G. (2010) · ACM Symposium on Applied Computing · doi:10.1145/1774088.1774591 · Cited by 19 · see citing works
Lin, C., Lee, C.-Y., & Wu, T. (2012) · International Journal of Security and Its Applications · Cited by 17 · see citing works
Implementing and Managing Policy Rules in Attribute Based Access Control
Hu, V. C., Ferraiolo, D. F., Kuhn, D. R., et al. (2015) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri.2015.98 · Cited by 16 · see citing works
TOWARDS A BRIGHT FUTURE: ENHANCING DIFFUSION OF CONTINUOUS CLOUD SERVICE AUDITING BY THIRD PARTIES
Lins, S., Teigeler, H., & Sunyaev, A. (2016) · European Conference on Information Systems · Cited by 16 · see citing works
Self-citationOn an agent-based architecture for collaborative continuous auditing
Wu, C.-H., Shao, Y. E., Ho, B.-Y., et al. (2008) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd.2008.4536957 · Cited by 15 · see citing works
Formalizing Theory Development in IS Design Science Research: Learning from Qualitative Research
Kuechler, B., Park, E. H., & Vaishnavi, V. (2009) · Journal of the Association for Information Systems · Cited by 15 · see citing works
Automated Audit of Compliance and Security Controls
Koschorreck, G. (2011) · doi:10.1109/imf.2011.12 · Cited by 15 · see citing works
LaBarge, R., & Mcguire, T. (2013) · Cited by 15 · see citing works
Towards Hybrid Cloud Service Certification Models
Katopodis, S., Spanoudakis, G., & Mahbub, K. (2014) · IEEE International Conference on Services Computing · doi:10.1109/scc.2014.59 · Cited by 14 · see citing works
Evaluating the Performance of Continuous Test-Based Cloud Service Certification
Stephanow, P., & Banse, C. (2017) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2017.134 · Cited by 14 · see citing works
Madison, M., Barnhill, M., Napier, C., et al. (2015) · Journal of international technology and information management · doi:10.58729/1941-6679.1032 · Cited by 13 · see citing works
A Test-Based Incremental Security Certification Scheme for Cloud-Based Systems
Anisetti, M., Ardagna, C. A., & Damiani, E. (2015) · IEEE International Conference on Services Computing · doi:10.1109/scc.2015.104 · Cited by 11 · see citing works
Technische Unterstützung von Audits bei Cloud-Betreibern
Kunz, T., Niehues, P., & Waldmann, U. (2013) · Datenschutz und Datensicherheit - DuD · doi:10.1007/s11623-013-0211-1 · Cited by 11 · see citing works
The technique of dynamic data masking in information systems
Baranchikov, A. I., Gromov, A. Y., Gurov, V. S., et al. (2016) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco.2016.7525695 · Cited by 9 · see citing works
Unblackboxing IT Certifications: A Theoretical Model Explaining IT Certification Effectiveness
Lins, S., & Sunyaev, A. (2017) · Journal of the Association for Information Systems · Cited by 9 · see citing works
Self-citationTeigeler, H., Lins, S., & Sunyaev, A. (2018) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2018.711 · Cited by 9 · see citing works
Self-citationResearch on Continuous Auditing Based on Multi-agent and Web Services
Ye, H., Yang, J., & Gan, Y. (2012) · International Conference on Management of e-Commerce and e-Government · doi:10.1109/icmecg.2012.73 · Cited by 6 · see citing works
Cloud Supply Chain Resilience Model: Development and Validation
Herrera, A., & Janczewski, L. J. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.489 · Cited by 6 · see citing works
Language Classes for Cloud Service Certification Systems
Stephanow, P., & Gall, M. (2015) · IEEE World Congress on Services (SERVICES) · doi:10.1109/services.2015.27 · Cited by 4 · see citing works
Design and Evaluation of Transfer-Supporting IT Components for Corporate Training Services
Amrou, S., & Boehmann, T. (2016) · International Conference on Information Systems · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Trustworthy artificial intelligence
Thiebes, S., Lins, S., & Sunyaev, A. (2021) · Electronic Markets · doi:10.1007/s12525-020-00441-4 · Cited by 595 · see citing works
Self-citation
- Authors
- Lins, S., Schneider, S., Szefer, J., Ibraheem, S., & Ali, A.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- Yale University (US)
- Venue
- Communications of the Association for Information Systems
- Publisher
- Association for Information Systems
- Published
- 2019
- Pages
- 406-510
- ISSN
- 1529-3181 (ISSN-L)
- OpenAlex
- W2937453880
- Cited by
- 26
- References
- 90
- Citations by year
- 2024: 3 · 2023: 7 · 2022: 5 · 2021: 3 · 2020: 3 · 2019: 5
- FWCI
- 4.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Communications of the Association for Information Systems (publishedVersion)
- Journal of the Association for Information Systems (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- cloud services, monitoring system, meta-requirements, design guidelines, certificate authority, cloud service providers, cloud infrastructure monitoring, expert interviews
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
