A problem-based approach to the advancement of heuristics for socio-technical evaluation

Abstract

With ubiquitous, mobile computing, health care systems, and smart factories, socio-technical phenomena continue to emerge that challenge traditional design and evaluation methods. We perceive such phenomena as the intertwinement of technical artifacts and social practices. Previous work shows that there is no sufficient method to evaluate the quality of this socio-technical intertwinement. Hence, our goal was to develop socio-technical heuristics, in short ST-heuristics, that can be applied by individuals to detect issues. Drawing inspiration from the success of usability heuristics in the field of human–computer interaction, we first applied a literature review to develop an initial set of ST-heuristics derived from six domains comprising groupware/computer-support cooperative work, job design, usability, socio-technical design principles, privacy, and process design. We then conducted two studies to evaluate and improve this set using empirical data from 13 cases from health care, industry, and engineering education fields. In total, we analysed 306 problems. The results substantiate a final set of eight ST-heuristics which allow for evaluating the socio-technical intertwinement in situ. We perceive the contribution of this work as a starting point for evaluators to uncover crucial issues and to improve current practice. We discuss the developed set of ST-heuristics within existing literature.