Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science
Open accessThe DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index; 9 of the 57 sources its claims cite were checked for support.
Abstract
To foster a sustainable society within a sustainable environment, we must dramatically reshape our work and consumption activities, most of which are facilitated through software. Yet, most software engineers hardly consider the effects on the sustainability of the IT products and services they deliver. This issue is exacerbated by a lack of methods and tools for this purpose. Despite the practical need for methods and tools that explicitly support consideration of the effects that IT products and services have on the sustainability of their intended environments, such methods and tools remain largely unavailable. Thus, urgent research is needed to understand how to design such tools for the IT community properly. In this article, we describe our experience using design science to create the Sustainability Awareness Framework (SusAF), which supports software engineers in anticipating and mitigating the potential sustainability effects during system development. More specifically, we identify and present the challenges faced during this process. The challenges that we have faced and addressed in the development of the SusAF are likely to be relevant to others who aim to create methods and tools to integrate sustainability analysis into their IT products and services development. Thus, the lessons learned in SusAF development are shared for the benefit of researchers and other professionals who design tools for that end.
Overview
CoChat Summary
TL;DR
This paper reports a decade-long design-science case study of developing the Sustainability Awareness Framework (SusAF) across 13 cycles, synthesizing lessons for building sustainability analysis tools in software engineering. It finds that SusAF broadened participants’ awareness of sustainability effects and identifies six lessons plus methodological limitations of design science for long-term, multidimensional sustainability impacts.
Key findings
- The SusAF development produced six lessons learned: need for methods/tools; need for dimensional and temporal link-up; need for examples; need for efficiency in elicitation and analysis; need for an introductory format; and need to adapt delivery to need or context.
- Industry workshops substantially broadened participants’ perception of sustainability effects: Datamatrix moved from 4 initially perceived effects to 153 potential effects; Jeppesen-Boeing went from 7 to 72 effects; Premier Park went from 2 to 144 effects, described as more than a 10-fold broadening.
- In teaching, the question set and improved artifacts were associated with more and better-contextualized effects, including an 80% increase in potential effects identified in LUT diagrams from academic year 2017–18 to 2018–19.
- Design science was useful for creating SusAF, but the paper identifies methodological concerns: timescale challenges for evaluating long-term sustainability impacts, scope and measurement challenges, and the anthropocentric focus of design science.
Methods
- Design: a design-science research project using Hevner’s three-cycle view—relevance, rigor, and design cycles—implemented over 13 cycles spanning almost a decade of SusAF development.
- Data and sample: 208 students from IT-related programs in the United States, Finland, Spain, Germany, and Sweden; 64 IT practitioners from 43 companies in Finland, India, the United States, Sweden, Germany, and Spain; plus an initial qualitative interview study with 13 RE practitioners from eight countries. Data included coursework, surveys, instructor/moderator reflections, SusADs, reports, and workshop feedback.
- Analysis: each cycle was analyzed separately to identify improvements and lessons learned; the paper reports descriptive statistics and qualitative feedback, with expert-panel/Delphi-style rounds used to develop the SusAF question set.
Limitations
- Paper-stated threats to validity include convenience sampling of students and companies, possible reactive bias among students, construct confounding, novelty effects, experimenter expectancies, limited external validity, and conclusion-validity concerns mitigated by participant diversity and shared facilitator processes.
- Paper-stated design-science limitations include difficulty evaluating long-term sustainability impacts within usual design-science timescales, lack of agreed units and boundaries for measuring social, individual, and environmental impacts, and design science’s anthropocentric focus.
- Evident from the text: this is a single-framework case study, and the authors state they cannot compare design science to any other method they have applied as extensively; long-term impact verification would require decades of data collection within the same context.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



![Fig. 4 Application of the design science research cycles to the development of the SusAF, adapted from Hevner and Chatterjee [51].](https://figures.cochat.ai/paper-figures/8b1b8a785d804b8ebb7483c5b84d2b24/lessons-learned-from-developing-a-sustainability-awareness-framework-for-figure-4.png)




![Fig. 9 Two pages from the workbook as thumbnails. The workbook is available for free [13].](https://figures.cochat.ai/paper-figures/8be6dda91db14ac69ce537e10325a827/lessons-learned-from-developing-a-sustainability-awareness-framework-for-figure-9.png)
Verification
Claims the paper makes
9 Valid · 3 Not valid · 34 Not checked · 136 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, the students were not sure as to how to initiate the process of sustainability requirements elicitation [35].”
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > Evaluation.
What the sources say
- Partly supportedcites [35] sourceExperimental — accuracy not yet measured
The closest our copy comes is 7.2 Method, which says the approach 'is not self-explanatory', that sufficient time must be set aside to describe it, that roleplay helped because 'the students could see how to use the framework', and that the question sets made students 'feel more confident when conducting the sustainability effect analysis' — i.e. students needed guidance and gained confidence, rather than a reported finding that they were unsure how to initiate sustainability requirements elicitation. The actually reported student difficulties are different in kind: 6.6.2 lists questions unrelated to the system (15%), lack of interviewee knowledge (10%) and question wording (9%), with one student finding it 'difficult to get a conversation going', and 7.3 identifies finding adequate surrogate stakeholders as the big challenge. 6.6.3 even records students seeing the questions as a 'good way to start' and the SusAD instructions as clear (87% liked them), so the claim's specific assertion of uncertainty about initiating the process overstates what this source states. (LLM, conf=0.68)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our observations corroborated other studies (e.g., [45,48,58]).”
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > 5.1 [Relevance] Cycle 1: Identified Need: Interview Study
What the sources say
- Not checked yetcites [45]
- Not checked yetcites [48] source
- Partly supportedcites [58] sourceExperimental — accuracy not yet measured
Our copy of Kirchherr et al. (2018) does report a widespread deficit of interest and awareness around sustainability/circular-economy thinking: Section 4.1 'Cultural Barriers' names 'Lacking consumer interest and awareness' as the most mentioned barrier among its 208 EU business and policy respondents, and Section 5 concludes the CE 'has not reached the mainstream yet'. That partly supports the citing sentence's idea of a corroborating study on limited understanding of sustainability. However, the paper's population is European consumers, businesses and policy-makers — it addresses neither students nor IT practitioners, and never discusses IT products or the relationship between sustainability and IT; searches for 'information technology / IT products / students' returned only reference-list entries (e.g. Barth et al., 2007 on higher education, cited for culture's role), not findings of this study. Moreover, Section 4 states explicitly that varying understandings of the CE concept among respondents was 'not the case', so an 'understanding' framing is not this paper's result. (LLM, conf=0.65)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“It is interesting to highlight that there was an 80% increase in the potential effects identified in LUT diagrams from the academic year 2017-18 to 2018-19 [34,35].”
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > Evaluation.
What the sources say
- Not checked yetcites [34] source
- Supportedcites [35] source
Section 6.4 'RQ4: identify more effects' states that with the question sheets there was 'a 10% increase in CSULB diagrams and an 80% increase in LUT diagrams' in the number of (potential) effects identified, comparing the spring 2019 SusADs with the spring 2018 baseline SusADs (Sects. 5.1-5.2 confirm the baseline was spring 2018 and the later round spring 2019, i.e. the 2017-18 to 2018-19 academic years at LUT and CSULB). The magnitude, the university (LUT), the artefact (LUT diagrams/SusADs) and the direction of change all match the citing sentence. (LLM, conf=0.90)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“IT practitioners do not commonly deal with effects that may unfold over time [8].”
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > 5.4 [Rigor and Design] Cycle 4: Development of Questions with Expert Panel State of the Art.
What the sources say
- Supportedcites [8] source
The Karlskrona Manifesto's Introduction (p. 2) states that 'software practice has a tendency to focus only on the immediate effects and tangible benefits of software products and platforms' and that 'the lack of long-term thinking in software engineering research and practice has been critiqued throughout the history of the discipline'. The 'We recognize that' section (p. 8) reinforces this: 'There is a tendency to focus on the immediate impacts of any new technology' and 'measuring success at one point in time fails to capture the effects that play out over multiple timescales'. This paraphrases the claim that practitioners do not commonly deal with effects unfolding over time; the only slight shift is 'IT practitioners' for the paper's 'software engineering research and practice'. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, presently, integration of sustainability analysis into software development is a very time consuming process.”
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > Implications for SE practice:
What the sources say
- Partly supportedcites [24] sourceExperimental — accuracy not yet measured
Section 4.2 (page unavailable) reports interviewees’ view that sustainability design requires extra work and an initial investment of time. Section 5.2 (page unavailable) says companies lack time and resources to commit to it. These passages support a time burden, but do not establish that integrating sustainability analysis is, as a general fact, a very time-consuming process. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
65 more claims
Create a free account to see all 182 claims and to verify the 48 references still open.
- Probably not a substantial claim. 74
- It may state two separate claims as one. 58
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bets et al.'s [18], Lagos et al.'s [98], and Naumann et al.'s [136] studies identify several potential indicators that need a large-scale benchmark evaluation and the integration within the ISO standards, like the environmental ISO 14,000 series and the social responsibility ISO 26000.”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > 6.3 Threats to Validity
“This study as described in Section 5 has more than 208 student participants from IT-related undergraduate and postgraduate courses across five countries, which”
§ Lessons Learned from Developing a Sustainability Awareness Framework for Software Engineering Using Design Science > 4.2 Participants
“Overall, we had 208 students in several countries-the United States (CSULB), Finland (LUT), Spain (LaSalle), Germany (HFU), and Sweden (KTH, LiU):”
Citations
Cites
(2002) · Cited by 13,211
Design Science in Information Systems Research1
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004) · MIS Quarterly · doi:10.2307/25148625 · Cited by 10,707
The Delphi method as a research tool: an example, design considerations and applications
Okoli, C., & Pawlowski, S. D. (2004) · Information & Management · doi:10.1016/j.im.2003.11.002 · Cited by 4,771 · cited 1×
“More specifically, we used an adaptation of the Delphi method [63, 77] to get these experts to define a set of questions to help start and guide the discussion on the effects of IT products on their in situ sustainability [35].”
An overview of the Schwartz theory of basic values
Schwartz, S. H. (2012) · Online Readings in Psychology and Culture · doi:10.9707/2307-0919.1116 · Cited by 4,147 · cited 1×
“…the current body of knowledge as a further input to the question set, including the Flourishing Business Canvas model [107], the Schwartz model of human values [104], the visualization diagram [8], the Sustainability Goal Reference model [87], and SIA (Societal Impact Assessment) literature.”
Doughnut Economics: Seven Ways to Think Like a 21st Century Economist
Diethelm, J. (2016) · She Ji: The Journal of Design, Economics, and Innovation · doi:10.1016/j.sheji.2017.05.002 · Cited by 2,071
Barriers to the Circular Economy: Evidence From the European Union (EU)
Kirchherr, J., Piscicelli, L., Bour, R., Kostense-Smit, E., Muller, J., Huibrechtse-Truijens, A., & Hekkert, M. (2018) · Ecological Economics · doi:10.1016/j.ecolecon.2018.04.028 · Cited by 1,903
Scenario types and techniques: Towards a user's guide
Börjeson, L., Höjer, M., Dreborg, K.-H., Ekvall, T., & Finnveden, G. (2006) · Futures · doi:10.1016/j.futures.2005.12.002 · Cited by 1,421 · cited 1×
“The “scenario techniques” widely used in future studies [16, 17] were also identified as a relevant method to help analyze and coherently present the possible future effects.”
Design Research in Information Systems: Theory and Practice
Hevner, A., & Chatterjee, S. (2010) · Integrated Series in Information Systems · doi:10.1007/978-1-4419-5653-8 · Cited by 1,382
Current validity of the Delphi method in social sciences
Landeta, J. (2006) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2005.09.002 · Cited by 1,353 · cited 1×
“More specifically, we used an adaptation of the Delphi method [63, 77] to get these experts to define a set of questions to help start and guide the discussion on the effects of IT products on their in situ sustainability [35].”
Design Science Methodology for Information Systems and Software Engineering
Wieringa, R. J. (2014) · Springer eBooks · doi:10.1007/978-3-662-43839-8 · Cited by 1,214 · cited 2×
“Design science research is a research paradigm that generates knowledge and scientific evidence by means of systematically developing an innovative artifact to solve human problems in a particular context [50, 51, 118].”
Agile software development: the business of innovation
Highsmith, J., & Cockburn, A. (2001) · Computer · doi:10.1109/2.947100 · Cited by 1,211 · cited 1×
“Modern software businesses must be agile and quick to market to be successful [53].”
The origins and evolution of scenario techniques in long range business planning
Bradfield, R., Wright, G., Burt, G., Cairns, G., & Van Der Heijden, K. (2005) · Futures · doi:10.1016/j.futures.2005.01.003 · Cited by 1,204 · cited 1×
“The “scenario techniques” widely used in future studies [16, 17] were also identified as a relevant method to help analyze and coherently present the possible future effects.”
Assessing ICT global emissions footprint: Trends to 2040 & recommendations
Belkhir, L., & Elmeligi, A. (2018) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2017.12.239 · Cited by 970 · cited 1×
“However, some scientists estimate that these will exceed 14% of the 2016-level worldwide greenhouse gas emissions by 2040 [10].”
The Assimilation of Software Process Innovations: An Organizational Learning Perspective
Fichman, R. G., & Kemerer, C. F. (1997) · Management Science · doi:10.1287/mnsc.43.10.1345 · Cited by 811 · cited 1×
“For example, a study of 608 IT organizations moving into object-oriented programming language concluded that IT practitioners were willing to try new technologies when knowledge could be acquired easily and cheaply, recommending proper packaging and support [40].”
The real climate and transformative impact of ICT: A critique of estimates, trends, and regulations
Freitag, C., Berners-Lee, M., Widdicks, K., Knowles, B., Blair, G. S., & Friday, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100340 · Cited by 570 · cited 1×
“This also applies to the IT sector and not simply because the current CO 2 emissions from the IT sector are already on par with that of the aviation industry (i.e., about 3% of global CO 2 emissions [43]).”
Upward, A., & Jones, P. (2016) · Organization & Environment · doi:10.1177/1086026615592933 · Cited by 512
Limits to Growth: The 30-Year Update
(2005) · International Journal of Sustainability in Higher Education · doi:10.1108/ijshe.2005.24906cae.006 · Cited by 465
ICT Innovations for Sustainability
(2015) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-09228-7 · Cited by 390
ICT for Sustainability: An Emerging Research Field
Hilty, L. M., & Aebischer, B. (2015) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-09228-7_1 · Cited by 388
A Model for Technology Transfer in Practice
Gorschek, T., Garre, P., Larsson, S., & Wohlin, C. (2006) · IEEE Software · doi:10.1109/ms.2006.147 · Cited by 350 · cited 1×
“Gorschek et al. [47] propose a seven-step technology transfer mode for ensuring close cooperation and collaboration between researchers and IT practitioners.”
Sustainability Design and Software: The Karlskrona Manifesto
Becker, C., Chitchyan, R., Duboc, L., Easterbrook, S., Penzenstadler, B., Seyff, N., & Venters, C. C. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.179 · Cited by 303 · cited 6×
“The SusAF is based on five dimensions [8] of sustainability, which are essentially an extension of the traditional three-dimensional view set out by the Brundtland Commission [120], which notes that environmental, economic, and societal dimensions together make up sustainability.”
InfluentialSelf-citationWamsler, C. (2020) · International Journal of Sustainability in Higher Education · doi:10.1108/ijshe-04-2019-0152 · Cited by 289
Garousi, V., Petersen, K., & Ozkan, B. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.07.006 · Cited by 241
Framing sustainability as a property of software quality
Lago, P., Koçak, S. A., Crnkovic, I., & Penzenstadler, B. (2015) · Communications of the ACM · doi:10.1145/2714560 · Cited by 229
Self-citationSoftware sustainability: Research and practice from a software architecture viewpoint
Venters, C. C., Capilla, R., Betz, S., Penzenstadler, B., Crick, T., Crouch, S., Nakagawa, E. Y., Becker, C., & Carrillo, C. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2017.12.026 · Cited by 201 · cited 1×
“The notion of sustainability has been discussed extensively in a number of publications, and readers are directed to these for an in-depth treatment of this topic [55, 62, 81, 108, 112, 113].”
Self-citationAn empirical study of maintenance and development estimation accuracy
Kitchenham, B., Lawrence Pfleeger, S., McColl, B., & Eagan, S. (2002) · Journal of Systems and Software · doi:10.1016/s0164-1212(02)00021-3 · Cited by 200 · cited 1×
“…empirical data was rather limited, researchers have employed expert panels to help elicit requirements and show that the expert knowledge reliably relates to the reality of the problem domain [9]—for example, from predicting requirements defects [64] to estimating software development effort [59].”
Requirements: The key to sustainability
Becker, C., Betz, S., Chitchyan, R., Duboc, L., Easterbrook, S. M., Penzenstadler, B., Seyff, N., & Venters, C. C. (2016) · IEEE Software · doi:10.1109/ms.2015.158 · Cited by 193 · cited 3×
“Based on the preceding literature, the first developed artifact of the SusAF was a central visualization diagram to discover, document, and validate potential sustainability effects [7].”
InfluentialSelf-citationDesign, science and wicked problems
Farrell, R., & Hooker, C. A. (2013) · Design Studies · doi:10.1016/j.destud.2013.05.001 · Cited by 179 · cited 1×
“In this paradigm, knowledge and understanding of a design problem and its solution are acquired through iterative development and evaluation of an artifact [38].”
A generic model for sustainability with process- and product-specific instances
Penzenstadler, B., & Femmer, H. (2013) · Proceedings of the 2013 workshop on Green in/by software engineering · doi:10.1145/2451605.2451609 · Cited by 173 · cited 1×
“…the current body of knowledge as a further input to the question set, including the Flourishing Business Canvas model [107], the Schwartz model of human values [104], the visualization diagram [8], the Sustainability Goal Reference model [87], and SIA (Societal Impact Assessment) literature.”
Self-citationUsing an expert panel to validate a requirements process improvement model
Beecham, S., Hall, T., Britton, C., Cottee, M., & Rainer, A. (2005) · Journal of Systems and Software · doi:10.1016/j.jss.2004.06.004 · Cited by 171
Value-Based Software Engineering: Overview and Agenda
Boehm, B. W. (2006) · Value-Based Software Engineering · doi:10.1007/3-540-29263-2_1 · Cited by 139 · cited 1×
“As a result, financial and technical planning, budgeting, monitoring, and maintaining tools have been developed and used for the well-attended facets [15, 78], resulting in so-called “sustainable software practices and processes” [42].”
Preventing Requirement Defects: An Experiment in Process Improvement
Lauesen, S., & Vinter, O. (2001) · Requirements Engineering · doi:10.1007/pl00010355 · Cited by 133 · cited 1×
“…empirical data was rather limited, researchers have employed expert panels to help elicit requirements and show that the expert knowledge reliably relates to the reality of the problem domain [9]—for example, from predicting requirements defects [64] to estimating software development effort [59].”
Understanding and improving technology transfer in software engineering
Pfleeger, S. L. (1999) · Journal of Systems and Software · doi:10.1016/s0164-1212(99)00031-x · Cited by 128 · cited 1×
“In literature, trialability and compatibility have been identified as key elements for the transfer of new solutions between researchers and IT practitioners in companies [89].”
Design science methodology: principles and practice
Wieringa, R. J. (2010) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1810295.1810446 · Cited by 112
There Is No Planet B: A Handbook for the Make or Break Years
Berners-Lee, M. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108545969 · Cited by 112
Software Engineering Education: Challenges and Perspectives
Ouhbi, S., & Pombo, N. (2020) · 2020 IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon45650.2020.9125353 · Cited by 106 · cited 1×
“In the literature, Ouhbi and Pombo [80] report that role-playing and problem-based learning are among the most effective ways of teaching SE.”
Sustainability design in requirements engineering: state of practice
Chitchyan, R., Becker, C., Betz, S., Duboc, L., Penzenstadler, B., Seyff, N., & Venters, C. C. (2016) · Proceedings of the 38th International Conference on Software Engineering Companion · doi:10.1145/2889160.2889217 · Cited by 104
Self-citationDesigning and Using Carbon Management Systems to Promote Ecologically Responsible Behaviors
Laval, U., & Corbett, J. (2013) · Journal of the Association for Information Systems · doi:10.17705/1jais.00338 · Cited by 88
Duboc, L., Penzenstadler, B., Porras, J., Akinli Kocak, S., Betz, S., Chitchyan, R., Leifler, O., Seyff, N., & Venters, C. C. (2020) · Requirements Engineering · doi:10.1007/s00766-020-00336-y · Cited by 88 · cited 10×
“Details of this evaluation were presented in the work of Duboc et al. [35].”
InfluentialSelf-citationKruchten, P., Nord, R. L., Ozkaya, I., & Falessi, D. (2013) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/2507288.2507326 · Cited by 87
From Computational Thinking to Systems Thinking: A conceptual toolkit for sustainability computing
Easterbrook, S. (2014) · Advances in Computer Science Research · doi:10.2991/ict4s-14.2014.28 · Cited by 85 · cited 2×
“Easterbrook [36] detected that software engineers and computer scientists, in general, are prone to learning much about computational thinking (i.e., how to divide and conquer challenges) but little about systems thinking (i.e., looking at the integrated big picture), which is crucial for…”
Towards a definition of sustainability in and for software engineering
Penzenstadler, B. (2013) · Proceedings of the 28th Annual ACM Symposium on Applied Computing · doi:10.1145/2480362.2480585 · Cited by 82
Self-citationReinerth, D., Busse, C., & Wagner, S. M. (2018) · Journal of Business Logistics · doi:10.1111/jbl.12190 · Cited by 75
Integrating sustainability in decision-making processes: A modelling strategy
Cabot, J., Easterbrook, S., Horkoff, J., Lessard, L., Liaskos, S., & Mazon, J.-N. (2009) · 2009 31st International Conference on Software Engineering - Companion Volume · doi:10.1109/icse-companion.2009.5070983 · Cited by 70 · cited 1×
“…traditional RE methods and tools do not explicitly facilitate the discussion of sustainability-related concerns, research suggests that existing RE techniques, approaches, and methods can serve as a starting point for practitioners to integrate sustainability into their practice [22, 25, 72, 105].”
Duboc, L., Betz, S., Penzenstadler, B., Akinli Kocak, S., Chitchyan, R., Leifler, O., Porras, J., Seyff, N., & Venters, C. C. (2019) · 2019 IEEE 27th International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00013 · Cited by 69 · cited 9×
“It is interesting to highlight that there was an 80% increase in the potential effects identified in LUT diagrams from the academic year 2017–18 to 2018–19 [34, 35].”
InfluentialSelf-citationCan You Hear the Rivers Sing? Legal Personhood, Ontology, and the Nitty-Gritty of Governance
Clark, C., Emmanouil, N., Page, J., & Pelizzon, A. (2019) · Cross Connect (Southern Cross University) · doi:10.15779/z388s4jp7m · Cited by 62 · cited 1×
“At a time when other disciplines are engaging with the representation and legal rights of non-human entities [26], this narrow anthropocentric approach is clearly inadequate.”
Klör, B., Monhof, M., Beverungen, D., & Bräuer, S. (2017) · European Journal of Information Systems · doi:10.1057/s41303-017-0044-3 · Cited by 61 · cited 1×
“For instance, Klör et al. [60] consider a decision support system that aids the automobile industry with repurposing electric vehicle batteries.”
Sustainable software engineering: Reflections on advances in research and practice
Venters, C. C., Capilla, R., Nakagawa, E. Y., Betz, S., Penzenstadler, B., Crick, T., & Brooks, I. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107316 · Cited by 59
Self-citationConsidering the social impacts of artefacts in information systems design science research
Leoz, G. D., & Petter, S. (2018) · European Journal of Information Systems · doi:10.1080/0960085x.2018.1445462 · Cited by 54 · cited 2×
“De Leoz and Petter [31] seek to raise awareness of IS design science research when considering the direct and indirect social effects of IS artifacts.”
Software engineering for sustainability: Find the leverage points!
Penzenstadler, B., Duboc, L., Venters, C. C., Betz, S., Seyff, N., Wnuk, K., Chitchyan, R., Easterbrook, S. M., & Becker, C. (2018) · IEEE Software · doi:10.1109/ms.2018.110154908 · Cited by 53 · cited 1×
“A few approaches have tried to take this cross-time and dimensional perspective [12, 19, 27, 75, 86, 105].”
Self-citationBeyond the Limits: Global Collapse or a Sustainable Future.
Common, M., Meadows, D. H., Meadows, D. L., & Randers, J. (1993) · The Economic Journal · doi:10.2307/2234737 · Cited by 52
Teaching sustainability: complexity and compromises
Anastasiadis, S., Perkiss, S., Dean, B. A., Bayerlein, L., González-Pérez, M. A., Wersun, A., Acosta, P., Jun, H. H., & Gibbons, B. (2020) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-02-2020-0029 · Cited by 52 · cited 1×
“Along the same line, Anastasiadis et al. [6] identified learning by doing, applying critical thinking, and fostering real-world engagement as crucial factors for teaching sustainability in its complexity.”
Saputri, T. R. D., & Lee, S.-W. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106407 · Cited by 50 · cited 1×
“Furthermore, Saputri and Lee [99] offer an integrated framework for requirements elicitation, the evaluation of which showed that participants could quickly identify more stakeholders, requirements, and features.”
Technology Transfer Barriers and Challenges Faced by R&D Organisations
Mazurkiewicz, A., & Poteralska, B. (2017) · Procedia Engineering · doi:10.1016/j.proeng.2017.03.134 · Cited by 47 · cited 1×
“Mazurkiewicz and Poteralska [69] observed that among the main barriers to technology transfer are companies’ focus on easily implemented technologies, difficulties in transmitting technical information from R&D organizations to the technology users, a high-level of tacit knowledge hindering the…”
An Interview Study on Sustainability Concerns in Software Development Projects
Groher, I., & Weinreich, R. (2017) · 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2017.70 · Cited by 45 · cited 1×
“Our observations corroborated other studies (e.g., [45, 48, 58]).”
An Action Research for Improving the Sustainability Assessment Framework Instruments
Condorí-Fernández, N., Lago, P., Luaces, M. R., & Places, Á. S. (2020) · Sustainability · doi:10.3390/su12041682 · Cited by 45 · cited 3×
“For example, the approach by Condori-Fernandez et al. [27] presents SAF, a decision-supporting tool for software architects.”
Defining the Concept of Sustainability - a Matter of Systems Thinking and Applied Systems Analysis
Sverdrup, H. U., & Svensson, M. (2004) · Springer eBooks · doi:10.1007/1-4020-2370-7_6 · Cited by 45 · cited 1×
“The notion of sustainability has been discussed extensively in a number of publications, and readers are directed to these for an in-depth treatment of this topic [55, 62, 81, 108, 112, 113].”
Characterising Sustainability Requirements: A New Species Red Herring or Just an Odd Fish?
Venters, C. C., Seyff, N., Becker, C., Betz, S., Chitchyan, R., Duboc, L., Mcintyre, D., & Penzenstadler, B. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Society Track (ICSE-SEIS) · doi:10.1109/icse-seis.2017.2 · Cited by 44 · cited 1×
“However, a careful look at the referenced literature suggests that the existing RE approaches are heavily biased toward sustainability goals related to second-order effects [111].”
Self-citationBrocke, J. V., & Seidel, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-29863-9_22 · Cited by 40 · cited 2×
“However, they consider only the “direct environmental impact due to their [products] physical existence . . . and their indirect impact on business processes”[20, p. 301].”
Samuel, M. (2010) · doi:10.1145/1709424.1709459 · Cited by 39
Sustainability requirement patterns
Roher, K., & Richardson, D. (2013) · 2013 3rd International Workshop on Requirements Patterns (RePa) · doi:10.1109/repa.2013.6602665 · Cited by 37 · cited 1×
“…into the SE process [97], the application of a sustainability requirement pattern to guide the specification of sustainability requirements [98], an economics-driven architectural evaluation method which extends the CBAM (Cost Benefits Analysis Method) and integrates principles of modern…”
Holopainen, J., Mattila, O., Pöyry, E., & Parvinen, P. M. T. (2020) · Forest Policy and Economics · doi:10.1016/j.forpol.2020.102190 · Cited by 34 · cited 1×
“As a poignant example, Holopainen et al. [56] describe the use of design science in the design of virtual reality services for forest management.”
Everything is INTERRELATED: teaching software engineering for sustainability
Penzenstadler, B., Betz, S., Venters, C. C., Chitchyan, R., Porras, J., Seyff, N., Duboc, L., & Becker, C. (2018) · IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) · doi:10.1145/3183377.3183382 · Cited by 29
Self-citationA proposed recommender system for eliciting software sustainability requirements
Roher, K., & Richardson, D. (2013) · 2013 2nd International Workshop on User Evaluations for Software Engineering Researchers (USER) · doi:10.1109/user.2013.6603080 · Cited by 27 · cited 1×
“…challenges, such as the use of a recommender system to overcome the barriers of incorporating sustainability into the SE process [97], the application of a sustainability requirement pattern to guide the specification of sustainability requirements [98], an economics-driven…”
Montgomery, L., & Damian, D. (2017) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2017.61 · Cited by 26 · cited 1×
“…framework [1], as well as characterizing the support process and data available to analysts in predicting the risk of escalating support tickets [74], a safety module for the Uni-REPM (Unified Requirements Engineering Process Maturity Model) [114], and a grid-based, service-oriented…”
Ojameruaye, B., Bahsoon, R., & Duboc, L. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2889160.2889218 · Cited by 24
Self-citationA classification approach of sustainability aware requirements methods
García‐Mireles, G. A., Moraga, M. Á., García, F. O., & Piattini, M. G. (2017) · Iberian Conference on Information Systems and Technologies (CISTI) · doi:10.23919/cisti.2017.7975813 · Cited by 24 · cited 1×
“…traditional RE methods and tools do not explicitly facilitate the discussion of sustainability-related concerns, research suggests that existing RE techniques, approaches, and methods can serve as a starting point for practitioners to integrate sustainability into their practice [22, 25, 72, 105].”
Tailoring Requirements Negotiation to Sustainability
Seyff, N., Betz, S., Duboc, L., Venters, C., Becker, C., Chitchyan, R., Penzenstadler, B., & Nobauer, M. (2018) · 2018 IEEE 26th International Requirements Engineering Conference (RE) · doi:10.1109/re.2018.00038 · Cited by 24
Self-citationDiscovering Requirements through Goal-Driven Process Mining
Dąbrowski, J., Kifetew, F. M., Muñante, D., Letier, E., Siena, A., & Susi, A. (2017) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew.2017.61 · Cited by 23
Monson, M. (2023) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1946442 · Cited by 21 · cited 2×
“Most recently, Monson [73] introduced a socially responsible design cience methodology encompassing “the objectives of socially inclusive and environmentally sustainable economic growth” (p. 1).”
Improving requirements-test alignment by prescribing practices that mitigate communication gaps
Bjarnason, E., Sharp, H., & Regnell, B. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09698-6 · Cited by 21 · cited 1×
“Bjarnason et al. [14] applied design science to the development of an SE improvement method, Gap Finder, that was designed to increase requirements-test alignment.”
Transformation²: Making software engineering accountable for sustainability
Schneider, C. M., & Betz, S. (2022) · Journal of Responsible Technology · doi:10.1016/j.jrt.2022.100027 · Cited by 21
Self-citationAn Exploratory Study on Technology Transfer in Software Engineering
Diebold, P., Vetro, A., & Mendez Fernandez, D. (2015) · 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) · doi:10.1109/esem.2015.7321189 · Cited by 17 · cited 1×
“They observed that transfer mediums are too often human intensive (e.g., personnel exchanges and meetings) rather than artifact based (e.g., tools, guidelines, and publications), yet it is the guidelines that are more often used by the industry partners [33].”
ShapeRE: Towards a Multi-Dimensional Representation for Requirements of Sustainable Software
Pham, Y. D., Bouraffa, A., & Maalej, W. (2020) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re48521.2020.00048 · Cited by 17 · cited 1×
“…sustainability concerns to architectural design decisions [76], a tool for requirement engineers to analyze the effects of the requirements on system sustainability [3], and a multi-dimensional framework for identifying requirements combined with a developer-oriented representation guideline [90].”
Safety Practices in Requirements Engineering: The Uni-REPM Safety Module
Vilela, J., Castro, J., Martins, L. E. G., & Gorschek, T. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2846576 · Cited by 17 · cited 1×
“…support process and data available to analysts in predicting the risk of escalating support tickets [74], a safety module for the Uni-REPM (Unified Requirements Engineering Process Maturity Model) [114], and a grid-based, service-oriented architecture for portfolio performance measurement [116].”
A concern-oriented sustainability approach
Brito, I. S., Conejero, J. M., Moreira, A., & Araújo, J. (2018) · International Conference on Research Challenges in Information Science (RCIS) · doi:10.1109/rcis.2018.8406676 · Cited by 16 · cited 1×
“A few approaches have tried to take this cross-time and dimensional perspective [12, 19, 27, 75, 86, 105].”
How Could We Have Known? Anticipating Sustainability Effects of a Software Product
Porras, J., Venters, C. C., Penzenstadler, B., Duboc, L., Betz, S., Seyff, N., Heshmatisafa, S., & Oyedeji, S. (2021) · Lecture notes in business information processing · doi:10.1007/978-3-030-91983-2_2 · Cited by 15 · cited 1×
“This work synthesizes the lessons learned throughout the development of the work published in five papers [24, 34, 35, 85, 91].”
Self-citationSuSoftPro: Sustainability Profiling for Software
Alharthi, A. D., Spichkova, M., & Hamilton, M. (2018) · 2018 IEEE 26th International Requirements Engineering Conference (RE) · doi:10.1109/re.2018.00072 · Cited by 14
Goal modeling for sustainability: The case of time
Mussbacher, G., & Nuttall, D. (2014) · 2014 IEEE 4th International Model-Driven Requirements Engineering Workshop (MoDRE) · doi:10.1109/modre.2014.6890821 · Cited by 13 · cited 2×
“A few approaches have tried to take this cross-time and dimensional perspective [12, 19, 27, 75, 86, 105].”
What Requirements Knowledge Do Developers Need to Manage Change in Safety-Critical Systems?
Goodrum, M., Cleland‐Huang, J., Lutz, R. R., Cheng, J., & Metoyer, R. (2017) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2017.65 · Cited by 13 · cited 1×
“For example, Goodrum et al [46] applied design science to aid in the discovery of the informational needs of five safety-critical system practitioners and 14 experienced developers as they engaged in software maintenance activities, then proposed and evaluated techniques for presenting and…”
Bridging the gap: SE technology transfer into practice: study design and preliminary results
Diebold, P., & Vetrò, A. (2014) · Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2652524.2652552 · Cited by 12
Knowledge Visualization: An Effective Way of Improving Learning
Zhang, J., Zhong, D., & Zhang, J. (2010) · International Workshop on Education Technology and Computer Science · doi:10.1109/etcs.2010.268 · Cited by 10
Designing a Conversational Requirements Elicitation System for End-Users
Rietz, T. (2019) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00061 · Cited by 10 · cited 1×
“…evolution of a range of RE approaches, including a perspective-based requirements methods checklist [30], a requirements self-elicitation system [96], an early-stage software quality requirements elicitation process [21], a quality-impact assessment method [41], and a regulator-oriented…”
Design Science, Engineering Science and Requirements Engineering
Wieringa, R. J., & Heerkens, H. (2008) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2008.63 · Cited by 9 · cited 1×
“Wieringa and Heerkens [119] state that there has been an ongoing debate in the computing sciences about the relative roles and merits of design versus empirical research and about the contribution of design and research methodology to the relevance of research results.”
Iterative Sustainability Impact Assessment: When to propose?
Penzenstadler, B., Betz, S., Duboc, L., Seyff, N., Porras, J., Oyedeji, S., Brooks, I., & Venters, C. C. (2021) · 2021 IEEE/ACM International Workshop on Body of Knowledge for Software Sustainability (BoKSS) · doi:10.1109/bokss52540.2021.00010 · Cited by 8 · cited 1×
“However, it can also be applied to evaluate and plan evolutionary changes [84].”
Self-citationSustainability-Aware Software Architecting for the Future Cloud
Fatima, I. (2023) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c57050.2023.00041 · Cited by 7
Review of Design-Oriented Green Information Systems Research
Brendel, A. B., Chasin, F., Mirbabaie, M., Riehle, D. M., & Harnischmacher, C. (2022) · Sustainability · doi:10.3390/su14084650 · Cited by 6
On the Veridicality of Claims in Design Science Research
Aljafari, R., & Khazanchi, D. (2013) · 2013 46th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2013.427 · Cited by 5 · cited 1×
“However, Aljafari and Khazanchi [4] highlight that the inability to objectively certify knowledge claims in design science research is a serious communication barrier that hinders the dissemination of knowledge across the different philosophical perspectives in the IS community and ultimately the…”
On Beyond Wicked: Exploring the Uses of "Wicked Problems"
Schuler, D. (2021) · LIMITS Workshop on Computing within Limits · doi:10.21428/bf6fb269.26fb2aec · Cited by 5 · cited 1×
“In addition, when dealing with the complex systems-related information and analyzing systems effects on sustainability, often abstract guidance is proposed [23, 103].”
Collaboration in sustainability vision
Mann, S., & Smith, L. (2011) · International Conference on Collaboration Technologies and Systems (CTS) · doi:10.1109/cts.2011.5928717 · Cited by 5 · cited 1×
“Similarly, Man et al. [67] and Mann and Smith [68] pointed out the need for computing to look at sustainability in a more holistic way.”
Betz, S., Duboc, L., Penzenstadler, B., Porras, J., Chitchyan, R., Seyff, N., Venters, C. C., & Brooks, I. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7342574 · Cited by 5
Self-citationA tale of three systems: Case studies on the application of architectural tactics for cyber-foraging
Lewis, G., Lago, P., Echeverría, S., & Simoens, P. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.01.052 · Cited by 4 · cited 2×
“The translation of interdisciplinary and time-dependent characteristics of sustainability into SE processes and tools still remain critical open research challenges today, despite the significant amount of related ongoing work (e.g., [34, 65]).”
Measuring Information Technology Service Levels: a Literature Review
Schorr, F., & Hvam, L. (2020) · IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) · doi:10.1109/ieem45057.2020.9309796 · Cited by 4 · cited 1×
“Schorr and Hvam [102] explored how IT managers can use design science to define and evaluate requirements for the information content of IT service catalogues in the early phase of the service design process.”
Weber, S., Beck, R., Wolf, M., & Vykoukal, J. (2010) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2010.469 · Cited by 2 · cited 1×
“…support process and data available to analysts in predicting the risk of escalating support tickets [74], a safety module for the Uni-REPM (Unified Requirements Engineering Process Maturity Model) [114], and a grid-based, service-oriented architecture for portfolio performance measurement [116].”
Towards a Regulator-Oriented Regulatory Intelligence Framework
Akhigbe, O. (2016) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2016.26 · Cited by 2
Daneva, M., Sikkel, K., Condorí-Fernández, N., & Herrmann, A. (2018) · Conducting Empirical Studies in Industry · doi:10.1145/3193965.3193966 · Cited by 2
The alliance between ecology and cybernetics for a new design science
Perriccioli, M. (2021) · TECHNE - Journal of Technology for Architecture and Environment · doi:10.13128/techne-9855 · Cited by 1 · cited 1×
“There is a demand for design approaches that recognize the complementarity of human and non-human agents [88].”
Betz, S., & Penzenstadler, B. (2024) · Zenodo · doi:10.5281/zenodo.10894317 · Cited by 1
Self-citationQuality-Impact Assessment of Software Systems
Fotrousi, F. (2016) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2016.53 · Cited by 1 · cited 1×
“…self-elicitation system [96], an early-stage software quality requirements elicitation process [21], a quality-impact assessment method [41], and a regulator-oriented intelligence framework [1], as well as characterizing the support process and data available to analysts in predicting…”
Using Design-science for the Representation of Non Functional Requirements
Ruiz, S. L. B., Pino, F. J., & Curieux, T. R. (2020) · Iberian Conference on Information Systems and Technologies (CISTI) · doi:10.23919/cisti49556.2020.9140876 · Cited by 1
Research works printed but not linked: 17
- Ian F. Alexander and Ljerka Beus-Dukic. 2009. Discovering Requirements: How to Specify Products and Services. Wiley.
- Stefanie Betz Christoph Becker Ruzanna Chitchyan Leticia Duboc Steve Easterbrook Birgit Penzenstadler Norbert Seyff and Colin Venters. 2015. Sustainability debt: A metaphor to support sustainability design decisions. In Proceedings of the International Workshop on Requirements Engineering for Sustainable Systems.
- Peter Checkland. 1999. Systems Thinking Systems Practice: Includes a 30-Year Retrospective. John Wiley & Sons Ltd.
- R. Chitchyan S. Betz L. Duboc B. Penzenstadler S. Easterbrook C. Ponsard and C. Venters. 2015. Evidencing sustainability design through examples. In Proceedings of the 4th International Workshop on Requirements Engineering for Sustainable Systems.
- Emelie Engström Margaret-Anne D. Storey Per Runeson Martin Höst and Maria Teresa Baldassarre. 2019. A review of software engineering research from a design science perspective. CoRR abs/1904.12742 (2019). http://arxiv.org/abs/1904.12742 · arXiv
- M. L. Gibson R. Chitchyan C. C. Venters M. Palacin-Silva L. Duboc B. Penzenstadler S. Betz and N. Seyff. 2017. Mind the chasm: A UK FishEye lens view of sustainable software engineering. In Proceedings of the 6th International Workshop on Requirements Engineering for Sustainable Systems (RE4SuSy’17). 15–24. http://eprints.hud.ac.uk/id/eprint/32429/
- André Heuer Philipp Diebold and Torsten Bandyszak. 2014. Supporting technology transfer by providing recommendations for writing structured guidelines. CEUR Workshop Proceedings 1129 (2014) 47–56.
- Alan Hevner. 2007. A three cycle view of design science research. Scandinavian Journal of Information Systems 19 (2007) 87–92.
- IPCC. 2022. Climate Change 2022: Impacts Adaptation and Vulnerability: Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press.
- Samuel Mann. 2011. The Green Graduate: Educating Every Student as a Sustainable Practitioner. NZCER Press Wellington New Zealand.
- Helena Holmström Olsson Hiva Alahyari and Jan Bosch. 2012. Climbing the “stairway to heaven”—A multiple-case study exploring barriers in the transition from agile development towards continuous deployment of software. In Proceedings of the 2012 38th Euromicro Conference on Software Engineering and Advanced Applications. IEEE 392–399.
- Alexander Osterwalder Yves Pigneur and Tim Clark. 2010. Business Model Generation: A Handbook for Visionaries Game Changers and Challengers. John Wiley & Sons Hoboken NJ.
- Birgit Penzenstadler Mara Anoushka Nam Stephanie and Budzinski Brian. 2019. Exploratory case study on sustainability awareness with a startup for business models. In Proceedings of the 6th International Conference on ICT for Sustainability (ICT4S’19).
- Roger S. Pressman. 1994. Software Engineering: A Practitioner’s Approach (European ed.). McGraw-Hill.
- Thorsten Schoormann Dennis Behrens and Ralf Knackstedt. 2018. Design principles for leveraging sustainability in business modelling tools. In Proceedings of the European Conference on Information Systems. https://api.semanticscholar.org/CorpusID:56140270
- World Commission on Environment and Development. 1987. Our Common Future. Oxford University Press, Oxford. http://www.un-documents.net/wced-ocf.htm
- 10.5555/1941925 · doi:10.5555/1941925
Cited by
Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights
Khan, T., Motie, S., Koçak, S. A., & Raza, S. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00067 · Cited by 25
“For instance, intersection of sustainability and software engineering is well established research area [3], [7], [26] such as exploring strategies for creating eco-friendly solutions that maintain functionality while reducing energy consumption [24].”
Sustainable Software Engineering: Concepts, Challenges, and Vision
König, C., Lang, D. J., & Schaefer, I. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709352 · Cited by 25
With Great Power Comes Great Responsibility: The Role of Software Engineers
Betz, S., & Penzenstadler, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715112 · Cited by 15
Self-citationA 2030 Roadmap for Software Engineering
Pezzè, M., Abrahão, S., Penzenstadler, B., Poshyvanyk, D., Roychoudhury, A., & Yue, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3731559 · Cited by 13
Self-citationde Walle, A. V., & Aerts, G. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103358 · Cited by 6
Metadata
- Authors
- Betz, S., Penzenstadler, B., Duboc, L., Chitchyan, R., Kocak, S. A., Brooks, I., Oyedeji, S., Porras, J., Seyff, N., & Venters, C. C.
- Affiliations
- Furtwangen University (DE)
- Chalmers University of Technology (SE)
- La Salle, University Ramon Llull, Barcelona, Spain
- University of Bristol (GB)
- Vector Institute for Artificial Intelligence, Toronto Canada
- University of the West of England (GB)
- Lappeenranta-Lahti University of Technology (FI)
- FHNW University of Applied Sciences and Arts Northwestern Switzerland (CH)
- University of Huddersfield (GB)
- ORCID
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2024-06-03
- Volume
- 33
- Issue
- 5
- Pages
- 1-39
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W4392593558
- Semantic Scholar
- 12fcd6110a8ed90fc001a166cef7a68bdf1875fc
- Cited by
- 23
- References
- 100
- Citations by year
- 2026: 12 · 2025: 8 · 2024: 3
- FWCI
- 2.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion, cc-by)
- Repository of Futwangen University of Applied Sciences (Furtwangen University) (submittedVersion, cc-by)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- UWE Research Repository (UWE Bristol) (submittedVersion, cc-by)
- Huddersfield Research Portal (University of Huddersfield) (submittedVersion, cc-by)
- Explore Bristol Research (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- Funders
- Departament de Recerca i Universitats of Catalunya (2021 SGR 01396)
- EPSRC (EP/R007373/1)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Green IT and Sustainability
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Engineering › Building and Construction › Sustainable Design and Development
- Keywords
- sustainability assessment, sustainability assessment tools, lessons learned, design science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.