The Semiotic Engineering of Human-Computer Interaction
Abstract
A theory of HCI that uses concepts from semiotics and computer science to focus on the communication between designers and users during interaction. In The Semiotic Engineering of Human-Computer Interaction, Clarisse Sieckenius de Souza proposes an account of HCI that draws on concepts from semiotics and computer science to investigate the relationship between user and designer. Semiotics is the study of signs, and the essence of semiotic engineering is the communication between designers and users at interaction time; designers must somehow be present in the interface to tell users how to use the signs that make up a system or program. This approach, which builds on—but goes further than—the currently dominant user-centered approach, allows designers to communicate their overall vision and therefore helps users understand designs—rather than simply which icon to click. According to de Souza's account, both designers and users are interlocutors in an overall communication process that takes place through an interface of words, graphics, and behavior. Designers must tell users what they mean by the artifact they have created, and users must understand and respond to what they are being told. By coupling semiotic theory and engineering, de Souza's approach to HCI design encompasses the principles, the materials, the processes, and the possibilities for producing meaningful interactive computer system discourse and achieves a broader perspective than cognitive, ethnographic, or ergonomic approaches. De Souza begins with a theoretical overview and detailed exposition of the semiotic engineering account of HCI. She then shows how this approach can be applied specifically to HCI evaluation and design of online help systems, customization and end-user programming, and multiuser applications. Finally, she reflects on the potential and opportunities for research in semiotic engineering.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If a representation is intended to convey meaning, that meaning only truly exists when generated by a viewer, listener, or user.
“If a representation (like a virtual exhibit) is intended to convey meaning, that meaning only truly exists when generated by a viewer, listener, or user [48].”
From Virtual Museums as Meaning-Modeling Systems in Digital Heritage (2025) · cites this paper as [48]
Meaning lies not in the content itself but in how that content is understood by someone.
“According to modeling theory, meaning lies not in the content itself but in how that content is understood by someone.”
From Virtual Museums as Meaning-Modeling Systems in Digital Heritage (2025) · cites this paper as [48]
Cited by
Sorted by citations
Tangible User Interfaces: Past, Present, and Future Directions
Shaer, O., & Hornecker, E. (2010) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000026 · Cited by 602 · see citing works
Visualization Rhetoric: Framing Effects in Narrative Visualization
Hullman, J. R., & Diakopoulos, N. A. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.255 · Cited by 505 · see citing works
“Semiotic theory has been introduced in HCI as an inspection method for interactive interfaces to help assess the designer-user meta-communication via the interactive artifact [17].”
MethodTangible User Interfaces: Past, Present, and Future Directions
Shaer, O. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601983299 · Cited by 414 · see citing works
A framework for analyzing and understanding online communities
de Souza, C. S., & Preece, J. J. (2004) · Interacting with Computers · doi:10.1016/j.intcom.2003.12.006 · Cited by 249 · see citing works
“de Souza, J. Preece / Interacting with Computers 16 (2004) 579–610 580”
BackgroundSelf-citationThe reification of metaphor as a design tool
Blackwell, A. F. (2006) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1188816.1188820 · Cited by 252 · see citing works
“These operational accounts of theory-based design might be founded in philosophy [Treglown 1999], applied linguistics [de Souza 2005], cognitive anthropology [Anderson et al. 1994], rhetoric [Marcus 1994], or indeed in any one of the disciplines that have constructed rich accounts of metaphor. HCI researchers wishing…”
BackgroundBorsci, S., Malizia, A., Schmettow, M., van der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2022) · Personal and Ubiquitous Computing · doi:10.1007/s00779-021-01582-9 · Cited by 266 · see citing works
“The ability to communicate and maintain an efficient and effective conversational exchange is not a secondary, but actually, a characterising element of chatbots that should be considered in the assessment of satisfaction with these tools [18, 19].”
BackgroundGuizzardi, G., Wagner, G., Almeida, J. P. A., et al. (2015) · Applied Ontology · doi:10.3233/ao-150157 · Cited by 236 · see citing works
“Loosely speaking, following the semiotic engineering principle of seeing representations as dialogues between designer and user (de Souza, 2005), we had the feeling that the users of the language were “speaking to us” via these “systematic subversions”.”
Video game values: Human–computer interaction and games
Barr, P., Noble, J., & Biddle, R. (2006) · Interacting with Computers · doi:10.1016/j.intcom.2006.08.008 · Cited by 226 · see citing works
Semiotic Engineering Methods for Scientific Research in HCI
de Souza, C. S., & Leitão, C. F. (2009) · Synthesis lectures on human-centered informatics · doi:10.2200/s00173ed1v01y200901hci002 · Cited by 209 · see citing works
Self-citationThe semiotic inspection method
de Souza, C. S., Leitão, C. F., Prates, R. O., et al. (2006) · doi:10.1145/1298023.1298044 · Cited by 190 · see citing works
Self-citationI did that! Measuring users' experience of agency in their own actions
Coyle, D., Moore, J., Kristensson, P. O., Fletcher, P. C., & Blackwell, A. F. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208350 · Cited by 174 · see citing works
“This understanding of the interface as the 'designer's deputy' [9] relies on the recognition of a design stance, in which the designer as an agent is recognised to have had a particular intent in the presentation of the product [8].”
BackgroundDesign as communication: exploring the validity and utility of relating intention to interpretation
Crilly, N., Good, D. A., Matravers, D., & Clarkson, P. J. (2008) · Design Studies · doi:10.1016/j.destud.2008.05.002 · Cited by 144 · see citing works
Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs
Subramonyam, H., Pea, R. D., Pondoc, C., Agrawala, M., & Seifert, C. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642754 · Cited by 169 · see citing works
“…conceived several diferent interaction paradigms (summarized in [65]), proposed frameworks to understand challenges in human-machine interactions [68, 120], and identifed ways to solve those challenges through the use of afordances, feedback mechanisms, task-oriented design, etc. [34, 53, 183].”
BackgroundVisual Interactive Systems for End-User Development: A Model-Based Design Methodology
Costabile, M. F., Fogli, D., Mussio, P., & Piccinno, A. (2007) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2007.904776 · Cited by 131 · see citing works
Crossing the bridge over Norman's Gulf of Execution
Vermeulen, J., Luyten, K., van den Hoven, E., & Coninx, K. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466255 · Cited by 131 · see citing works
Crilly, N., Maier, A. M., & Clarkson, P. J. (2008) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 105 · see citing works
Preece, J. J. (2004) · Communications of the ACM · doi:10.1145/975817.975845 · Cited by 94 · see citing works
Ontology, Ontologies and the “I” of FAIR
Guizzardi, G. (2019) · Data Intelligence · doi:10.1162/dint_a_00040 · Cited by 92 · see citing works
Can inspection methods generate valid new knowledge in HCI? The case of semiotic inspection
de Souza, C. S., Leitão, C. F., Prates, R. O., Bim, S. A., & da Silva, E. J. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.08.006 · Cited by 93 · see citing works
“Semiotic engineering is a semiotic theory of HCI (de Souza, 2005).”
BackgroundInfluentialSelf-citationCollective intelligence in law enforcement – The WikiCrimes system
Furtado, V., Ayres, L., de Oliveira, M., Vasconcelos, E., Caminha, C., D’Orleans, J., & Belchior, M. (2009) · Information Sciences · doi:10.1016/j.ins.2009.08.004 · Cited by 92 · see citing works
Critical methods and user generated content
Blythe, M., & Cairns, P. A. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518923 · Cited by 87 · see citing works
Mapping Candidate Within-Vehicle Auditory Displays to Their Referents
McKeown, D., & Isherwood, S. J. (2007) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x200067 · Cited by 80 · see citing works
“Semiotics has been centrally placed in the theoretical development and empirical investigation of visual icons, with human-computer interaction (HCI) benefiting from notable theoretical guidelines (de Souza, 2005).”
BackgroundAn integrative semiotic framework for information systems: The social, personal and material worlds
Mingers, J., & Willcocks, L. P. (2014) · Information and Organization · doi:10.1016/j.infoandorg.2014.01.002 · Cited by 79 · see citing works
“De Souza (2005) has developed a theory of semiotic engineering which sees HCI as enabling an active communication process between the system user and (implicitly) the system designer, and O’Neill (2008) has built on these ideas.”
BackgroundSemiotic engineering: bringing designers and users together at interaction time
de Souza, C. S. (2005) · Interacting with Computers · doi:10.1016/j.intcom.2005.01.007 · Cited by 79 · see citing works
“Our aim with this paper is to show how semiotic engineering—a semiotic theory of HCI (de Souza et al., 2001; de Souza, 2005), can contribute to bring about new perceptions and design possibilities in HCI.”
BackgroundInfluentialSelf-citationCommunicability of traditional interfaces VS chatbots in healthcare and smart home domains
Valtolina, S., Barricelli, B. R., & Gaetano, S. D. (2019) · Behaviour and Information Technology · doi:10.1080/0144929x.2019.1637025 · Cited by 77 · see citing works
“For measuring the communicability of a system, we adopted a Communicability Evaluation Method (CEM), which is based on the semiotic engineering theory (De Souza 2005).”
MethodAntonijević, S. (2008) · Information Communication & Society · doi:10.1080/13691180801937290 · Cited by 76 · see citing works
Towards user–intuitive web interface sign design and evaluation: A semiotic framework
Islam, M. N., & Bouwman, H. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2015.10.003 · Cited by 73 · see citing works
Young People's Engagement with Digital Literacies in Marginal Contexts in a Globalised World
Snyder, I., & Prinsloo, M. (2007) · Language and Education · doi:10.2167/le745.0 · Cited by 70 · see citing works
Putting Social Context into Text: The Semiotics of E‐mail Interaction
Menchik, D. A., & Tian, X. (2008) · American Journal of Sociology · doi:10.1086/590650 · Cited by 66 · see citing works
Places: People, Events, Loci – the Relation of Semantic Frames in the Construction of Place
Harrison, S., & Tatar, D. G. (2007) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-007-9073-0 · Cited by 58 · see citing works
“Clarisse de Souza’s recent book, The Semiotic Engineering of Human-Computer Interaction, takes this one step further by discussing how HCI design can create consistent meaning in an interface (de Souza 2005).”
BackgroundVideogame analysis: a social-semiotic approach
Pérez-Latorre, Ó., Oliva, M., & Besalú, R. (2016) · Social Semiotics · doi:10.1080/10350330.2016.1191146 · Cited by 58 · see citing works
Spaces of Interaction, Places for Experience
Benyon, D. (2014) · Synthesis lectures on human-centered informatics · doi:10.1007/978-3-031-02206-7 · Cited by 63 · see citing works
An integrative semiotic methodology for IS research
Mingers, J., & Willcocks, L. P. (2017) · Information and Organization · doi:10.1016/j.infoandorg.2016.12.001 · Cited by 54 · see citing works
Evolution, Choice, and Scaffolding: Semiosis is Changing Its Own Building
Kull, K. (2015) · Biosemiotics · doi:10.1007/s12304-015-9243-2 · Cited by 53 · see citing works
Formative Interfaces for Scaffolding Self-Regulated Learning in PLEs
Türker, M. A., & Zingel, S. (2008) · E-learning papers · Cited by 49 · see citing works
Barbosa, S. D. J., Silveira, M. S., & Gasparini, I. (2016) · Scientometrics · doi:10.1007/s11192-016-2162-4 · Cited by 49 · see citing works
Designing for collaboration: a study in intergenerational social game design
Derboven, J., Gils, M. V., & Grooff, D. D. (2011) · Universal Access in the Information Society · doi:10.1007/s10209-011-0233-0 · Cited by 48 · see citing works
“For instance, De Souza [6, 23] and the Semiotic Engineering Research Group (SERG) have been using tagging in ‘‘communica-bility analyses’’ to trace communicative breakdowns during interaction, categorizing observable user utterances and interactions.”
MethodOrganizational Affordances: A Structuration Theory Approach to Affordances
Vyas, D., Chisalita, C. M., & Dix, A. J. (2016) · Interacting with Computers · doi:10.1093/iwc/iww008 · Cited by 46 · see citing works
“Very recently, Sun and Hart-Davidson (2014) developed a ‘relational approach’ to study affordances.”
BackgroundProcessing and Visualizing Data in Complex Learning Environments
Thompson, K., Ashe, D., Carvalho, L., Goodyear, P., Kelly, N., & Parisio, M. L. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479368 · Cited by 44 · see citing works
Valério, F. A. M., Guimarães, T. G., Prates, R. O., & Candello, H. C. D. S. P. (2017) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3160504.3160544 · Cited by 44 · see citing works
“The Semiotic Inspection Method (SIM) [8] is a qualitative evaluation method based on Semiotic Engineering [7] that assesses systems’ communicability.”
BackgroundMethodSemiotic Inspection of a Game for Children with Down Syndrome
Brandão, A. A. P., Trevisan, D. G., Brandão, L., et al. (2010) · doi:10.1109/sbgames.2010.24 · Cited by 43 · see citing works
Bardzell, J., & Bardzell, S. (2008) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1358628.1358703 · Cited by 42 · see citing works
Zhao, S., & van Leeuwen, T. (2014) · Classroom Discourse · doi:10.1080/19463014.2013.859848 · Cited by 41 · see citing works
An artifact ontology for design science research
Weigand, H., Johannesson, P., & Andersson, B. (2021) · Data & Knowledge Engineering · doi:10.1016/j.datak.2021.101878 · Cited by 40 · see citing works
Graphics and Semantics: The Relationship between What Is Seen and What Is Meant in Icon Design
Isherwood, S. J. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02728-4_21 · Cited by 40 · see citing works
“There is not necessarily a direct connection between the Object and Representamen [5].”
BackgroundBlythe, M. (2013) · Personal and Ubiquitous Computing · doi:10.1007/s00779-013-0679-6 · Cited by 38 · see citing works
The sense of the interface: Applying semiotics to HCI research
Scolari, C. A. (2009) · Semiotica · doi:10.1515/semi.2009.067 · Cited by 38 · see citing works
Monitoring and Detection of Agitation in Dementia: Towards Real-Time and Big-Data Solutions
Moore, P. T., Xhafa, F., Barolli, L., & Thomas, A. (2013) · International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC) · doi:10.1109/3pgcic.2013.26 · Cited by 37 · see citing works
Blackwell, A. F. (2015) · Aarhus Series on Human Centered Computing · doi:10.7146/aahcc.v1i1.21197 · Cited by 35 · see citing works
“In symbolic systems, where the system behavior has been specified and encoded directly by a human designer, the user can apply a semiotic reading in which the user interface acts as the ‘designer’s deputy’ [34].”
BackgroundDesign da interação humano-computador com MoLIC
Barbosa, S. D. J., & da Silva, B. S. (2014) · Human Factors in Computing Systems · Cited by 34 · see citing works
Social tagging, online communication, and Peircean semiotics: a conceptual framework
Huang, A. W., & Chuang, T. (2009) · Journal of Information Science · doi:10.1177/0165551508099606 · Cited by 34 · see citing works
“If we accept de Souza’s view of semiotic engineering [10], the explanation of why general social taggers act like light-weight system designers is clearly mirrored in the semiotic analysis of meta-communication between machines, users and designers.”
BackgroundInfluentialEvaluating Web and Mobile User Interfaces With Semiotics: An Empirical Study
Islam, M. N., Bouwman, H., & Islam, A. K. M. N. (2020) · IEEE Access · doi:10.1109/access.2020.2991840 · Cited by 34 · see citing works
“De Souza [16], for instance, introduced a theory of semi-otic engineering designed to analyze the connection between system signs (e.g., links, icons, buttons, etc.), semantics and functions, with the aim of understanding the meta-communication between designers and users [16], [17], [20],…”
BackgroundMethodInfluentialAppropriating virtual learning environments: A study of teacher tactics
Derboven, J., Geerts, D., & Grooff, D. D. (2017) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2017.01.002 · Cited by 33 · see citing works
“In this literature, semiotics has been used to inform the design process (e.g., Liu, 2000), to guide designers in communicating their intentions clearly, or to evaluate communicative clarity (e.g., De Souza, 2005).”
MethodA systematic literature review of semiotics perception in user interfaces
Islam, M. N. (2013) · Journal of Systems and Information Technology · doi:10.1108/13287261311322585 · Cited by 33 · see citing works
GranDIHC-BR 2025-2035 - GC1: New Theoretical and Methodological Approaches in HCI
Junior, D. P. D. S., Alves, D. D., Carneiro, N., Matos, E. D. S., Baranauskas, M. C. C., & Mendoza, Y. L. M. (2024) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3702038.3702054 · Cited by 33 · see citing works
“Semiotic Engineering [17, 47, 48, 50], for example, is a Semiotic theory [63, 132] of HCI to inform and support the design and evaluation of interactive systems from the perspective of meaning and communication.”
Artificial Intelligence + Distributed Systems = Agents
Dziţac, I., & Bărbat, B. E. (2009) · International Journal of Computers Communications & Control · doi:10.15837/ijccc.2009.1.2410 · Cited by 32 · see citing works
“…of purposes, sign producers (i.e., signification system users in this specific role) choose to express intended meanings by exploring the possibilities of existing signification systems or, occasionally, by resorting to non-systematized signs, which they invent or use in unpredicted ways” [19].”
BackgroundPerceiving images and telling tales: A visual and verbal analysis of the meaning of the internet
Joy, A., Sherry, J. F., Venkatesh, A., & Deschênes, J. (2009) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2009.05.013 · Cited by 29 · see citing works
“However, themost dominant metaphors for the Internet have been either spatio-temporal (e.g. MySpace, the world wide web, Explorer) where people “navigate” or “surf the web” or communication oriented (MUApp) emphasizing virtual interactions (de Souza, 2005).”
BackgroundSemio-Participatory Framework for Interaction Design of Educational Software
Rosa, J. C. S., & Matos, E. D. S. (2016) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3033701.3033734 · Cited by 30 · see citing works
Applicability of the semiotic inspection method: a systematic literature review
Reis, S., & Prates, R. O. (2011) · Latin American conference on Human-Computer Interaction · doi:10.5555/2254436.2254468 · Cited by 28 · see citing works
The To-and-Fro of Sense Making
Hornecker, E. (2016) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2882785 · Cited by 27 · see citing works
“Semiotic analysis and semiotics-informed design of user interfaces has thus become a field of its own [de Souza 2005].”
MethodCarvalho, F., Trevisan, D. G., & Raposo, A. B. (2011) · Virtual Reality · doi:10.1007/s10055-011-0205-y · Cited by 27 · see citing works
“Signification is the process through which certain systems of signs are established by virtue of social and cultural conventions adopted by the users (De Souza 2005).”
BackgroundParticipatory knowledge-management design: A semiotic approach
Valtolina, S., Barricelli, B. R., & Dittrich, Y. (2011) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2011.11.007 · Cited by 27 · see citing works
Using an interaction model as a resource for communication in design
de Paula, M. G., da Silva, B. S., & Barbosa, S. D. J. (2005) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1056808.1057004 · Cited by 26 · see citing works
“This work is grounded on semiotic engineering, a theory of HCI that characterizes human-computer interaction phenomena and provides an ontology from which HCI frameworks and models can be derived [3].”
BackgroundA Semiotic Framing for End-User Development
de Souza, C. S., & Barbosa, S. D. J. (2006) · Human-computer interaction series · doi:10.1007/1-4020-5386-x_18 · Cited by 26 · see citing works
Self-citationIslam, M. N., Khan, N. I., Inan, T. T., & Sarker, I. H. (2023) · SAGE Open · doi:10.1177/21582440231172741 · Cited by 26 · see citing works
“The semiotic theory and the concept of semiotic engineering are also widely used to design intuitive interfaces for end-users (De Souza, 2005; Islam, 2013).”
BackgroundBarbosa, S. D. J., Barbosa, G. D. J., de Souza, C. S., & Leitão, C. F. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445900 · Cited by 25 · see citing works
“In this section we briefly introduce the key points of the foundational theory for our research, Semiotic Engineering [12] (subsection 2.1), and comment on background work that, like Coeckel-bergh’s [11], has provided us with insights on either the problem we want to address, or the solution we…”
BackgroundMethodInfluentialSelf-citationMaciel, C. D. O., & Pereira, V. C. (2012) · Human Factors in Computing Systems · doi:10.5555/2393536.2393548 · Cited by 25 · see citing works
de Melo, D. N. A., & Monteiro, I. T. (2021) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3472301.3484362 · Cited by 25 · see citing works
“O MIS é um método de base qualitativa baseado na Engenharia Semiótica [17] que avalia a comunicabilidade dos sistemas.”
BackgroundJones, R. H., Chik, A., & Hafner, C. A. (2015) · Routledge eBooks · doi:10.4324/9781315726465-1 · Cited by 23 · see citing works
An assessment of a semiotic framework for evaluating user-intuitive Web interface signs
Islam, M. N., & Bouwman, H. (2015) · Universal Access in the Information Society · doi:10.1007/s10209-015-0403-6 · Cited by 23 · see citing works
“De Souza [10, 15] introduced the theory of semiotic engineering in HCI.”
BackgroundMethodInfluentialAssessing the usability of e-government websites of Bangladesh
Islam, M. N., Rahman, S., & Islam, M. S. (2017) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecace.2017.7913026 · Cited by 23 · see citing works
“In nutshell, we hope that our results will be used by UI designers and usability evaluators and taken up by other HCI and semiotic researchers.”
BackgroundInfluentialMEANING IN PRODUCT USE: A DESIGN PERSPECTIVE
Boess, S. U., & Kanis, H. (2008) · Elsevier eBooks · doi:10.1016/b978-008045089-6.50015-0 · Cited by 23 · see citing works
Borsci, S., & Schmettow, M. (2024) · Personal and Ubiquitous Computing · doi:10.1007/s00779-024-01834-4 · Cited by 24 · see citing works
“…by users after the interaction with interactive systems; nevertheless, these scales were not meant to capture the aspects associated with the textual or verbal dialogical exchanges, i.e., the ability to maintain the sense of the conversation and its context in an efficient and effective [30, 31].”
Developing heuristics for the semiotics inspection of websites
Bolchini, D., Chatterji, R., & Speroni, M. (2009) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/1621995.1622009 · Cited by 22 · see citing works
Move, Hold and Touch: A Framework for Tangible Gesture Interactive Systems
Angelini, L., Lalanne, D., van den Hoven, E., Khaled, O. A., & Mugellini, E. (2015) · Machines · doi:10.3390/machines3030173 · Cited by 22 · see citing works
“Syntactics and semantics are two of the three branches of semiotics, which studies signs in different domains, with some examples also in HCI [22,30].”
BackgroundSemantic Knowledge Management for Education
Kohlhase, A. E., & Kohlhase, M. (2008) · Proceedings of the IEEE · doi:10.1109/jproc.2008.921606 · Cited by 22 · see citing works
“In the example, the observer is a piece of software that represents the designer as her deputy [19].”
BackgroundMaking Decisions About Digital Legacy with Google’s Inactive Account Manager
Prates, R. O., Rosson, M. B., & de Souza, C. S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22701-6_14 · Cited by 21 · see citing works
“SIM is a method of Semiotic Engineering, a semiotic theory of HCI that views human-computer interaction as a special case of computer-mediated human communication involving a system’s designers and users [2].”
MethodSelf-citationde Souza, A. A., Merino, E., da Silva, L. H. G., Muñoz, R., Guiñez, R., Villarroel, R., & Barcelos, T. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.3217/jucs-024-02-0149 · Cited by 21 · see citing works
de Souza, C. S., Nicolaci-da-Costa, A. M., da Silva, E. J., et al. (2004) · Interacting with Computers · doi:10.1016/j.intcom.2004.07.003 · Cited by 21 · see citing works
Self-citationAn overview of the main design characteristics of simulation games in Software Engineering education
Peixoto, D. C. C., Possa, R. M., Resende, R. F., et al. (2011) · doi:10.1109/cseet.2011.5876076 · Cited by 21 · see citing works
Coutinho, F., Prates, R. O., & Chaimowicz, L. (2011) · doi:10.1109/sbgames.2011.16 · Cited by 21 · see citing works
“According to [23], communication is achieved in software interfaces by codification and later interpretation of signs (anything that represents something to someone).”
BackgroundMethodInfluentialCultural theory and real world design
Satchell, C. (2008) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1357054.1357303 · Cited by 21 · see citing works
“On a more specific level, there is use of the individual components of cultural theory within HCI such as Marxism [35], feminism [8], semiotics [10] and hermeneutics [45].”
BackgroundHow digital visualizations shape strategy work on the frontlines
Azad, B., & Zablith, F. (2020) · Long Range Planning · doi:10.1016/j.lrp.2020.101990 · Cited by 20 · see citing works
“We adopted a think-aloud technique (De Souza, 2005; Prates et al., 2000), and encouraged the participants to express and be verbose about their actions while working on the given task.”
MethodFerreira, J. J., de Souza, C. S., Salgado, L. C. D. C., et al. (2012) · Proceedings/Proceedings -- IEEE Symposium on Visual Languages and Human-Centric Computing · doi:10.1109/vlhcc.2012.6344492 · Cited by 20 · see citing works
“This hybrid set of data was analyzed using a combination of three methods: discourse analysis (DA) for interview and verbal protocol data; Semiotic Engineering inspection methods and concepts (SIM) for AgentSheets designer-user communication signs; and the Cognitive Dimensions of Notations (CDN) framework for visual p…”
MethodInfluentialSelf-citationOn the epistemic nature of cultural viewpoint metaphors
Salgado, L. C. D. C., de Souza, C. S., & Leitão, C. F. (2011) · Latin American conference on Human-Computer Interaction · doi:10.5555/2254436.2254443 · Cited by 20 · see citing works
Self-citationIntegrating Semiotics Perception in Usability Testing to Improve Usability Evaluation
Islam, M. N., & Tétard, F. (2013) · Advances in human and social aspects of technology book series · doi:10.4018/978-1-4666-4046-7.ch007 · Cited by 19 · see citing works
Chatbots Explain Themselves: Designers' Strategies for Conveying Chatbot Features to Users
Valério, F. A. M., Guimarães, T. G., Prates, R. O., & Candello, H. C. D. S. P. (2018) · Journal on Interactive Systems · doi:10.5753/jis.2018.710 · Cited by 19 · see citing works
“The property that defines how well the system communicates to users the designer’s intentions and underlying principles has been defined as communicability [7], [8]).”
BackgroundMethodSoftwares sociais: uma visão orientada a valores
Pereira, R., Baranauskas, M. C. C., & da Silva, S. R. P. (2010) · Human Factors in Computing Systems · doi:10.5555/1999593.1999610 · Cited by 22 · see citing works
Critical Issues in Interaction Design
Blythe, M., Bardzell, J., Bardzell, S., et al. (2008) · Electronic workshops in computing · doi:10.14236/ewic/hci2008.67 · Cited by 19 · see citing works
Semiotic inspection method in the context of educational simulation games
Peixoto, D. C. C., Prates, R. O., & Resende, R. F. (2010) · ACM Symposium on Applied Computing · doi:10.1145/1774088.1774342 · Cited by 19 · see citing works
“There are two distinct methods to evaluate the communicability of an interface: Semiotic Inspection Method (SIM) [4], [5], [6], [7] and Communicability Evaluation Method (CEM) [4], [15].”
BackgroundMethodInfluentialEvaluating HCI Design with Interaction Modeling and Mockups - A Case Study
Lopes, A., Marques, A. B., Barbosa, S. D. J., et al. (2015) · International Conference on Enterprise Information Systems · doi:10.5220/0005374200790087 · Cited by 18 · see citing works
“MoLIC is grounded in Semiotic Engineering (De Souza, 2005), a theory of HCI with particular focus on communication between the designer1 and the user mediated by interactive systems.”
BackgroundAn Interaction Modeling Language for Therapeutic Applications
Garcia, F. E., Rodrigues, K. R. D. H., & Néris, V. P. D. A. (2016) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3033701.3033733 · Cited by 18 · see citing works
“A linguagem usa a Engenharia Semiótica [6] como fundamen-tação teórica, pois essa caracteriza a interação usuário-sistema como uma conversa entre usuários e designers através da interface do usuário [23].”
A data-driven computational semiotics: The semantic vector space of Magritte’s artworks
Chartier, J.-F., Pulizzotto, D., Chartrand, L., & Meunier, J.-G. (2019) · Semiotica · doi:10.1515/sem-2018-0120 · Cited by 18 · see citing works
Social software building blocks: Revisiting the honeycomb framework
Pereira, R., Baranauskas, M. C. C., & da Silva, S. R. P. (2010) · International Conference on Information Society · doi:10.1109/i-society16502.2010.6018707 · Cited by 18 · see citing works
Notes on a Linguistic Description as the Basis for Automatic Image Understanding
Tadeusiewicz, R., Ogiela, M. R., & Szczepaniak, P. S. (2009) · International Journal of Applied Mathematics and Computer Science · doi:10.2478/v10006-009-0013-7 · Cited by 18 · see citing works
Usability Practice: The Appealing Way to HCI
Rusu, C., Rusu, V. Z., & Roncagliolo, S. (2008) · doi:10.1109/achi.2008.14 · Cited by 18 · see citing works
“of semiotic engineering [2], [6], [7], [8].”
Kundu, S., Kabir, A., & Islam, M. N. (2020) · IEEE Region Humanitarian Technology Conference · doi:10.1109/r10-htc49770.2020.9357061 · Cited by 17 · see citing works
“De Souza proposed a theory of semiotic engineering to explore the connection among system signs (e.g., links, icons, buttons, etc.), semantics, and functions, for realizing the communication between designers and users [17]–[19].”
BackgroundAuditory Displays as Occasion Setters
McKeown, D., Isherwood, S. J., & Conway, G. E. (2010) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810366861 · Cited by 17 · see citing works
“Therefore, the information must be encoded into a “sound sign” (as opposed to the usual visual signs or icons within the semiotics literature; cf. de Souza, 2005).”
BackgroundCarrilho, J. M., de Oliveira, I. J. R., Santos, D., Osanan, G. C., Cruz-Correia, R. J., & Reis, Z. S. N. (2018) · JMIR Formative Research · doi:10.2196/11374 · Cited by 17 · see citing works
“Proposed by de Souza (2005), this evaluation allows the identification of communication breakdown with the computational artifact during user interaction [16].”
Turn off the graphics: designing non-visual interfaces for mobile phone games
Valente, L., de Souza, C. S., & Feijó, B. (2009) · Journal of the Brazilian Computer Society · doi:10.1007/bf03192576 · Cited by 17 · see citing works
“53 Turn off the graphics: designing non-visual interfaces for mobile phone games 2009; 15(1)”
BackgroundMethodInfluentialSelf-citation
- Authors
- de Souza, C. S.
- Venue
- The MIT Press eBooks
- Publisher
- The MIT Press
- Published
- 2005-02-11
- OpenAlex
- W2117812870
- Cited by
- 757
- References
- 0
- Citations by year
- 2026: 19 · 2025: 25 · 2024: 19 · 2023: 33 · 2022: 30 · 2021: 29 · 2020: 30 · 2019: 33 · 2018: 39 · 2017: 45 · 2016: 58 · 2015: 61 · 2014: 36 · 2013: 67 · 2012: 39
- FWCI
- 22.30
- Type
- book
- Language
- en
- Open access
- closed
- Locations
- The MIT Press eBooks (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- semiotic engineering, human-computer interaction theory, semiotics, designer-user communication, user-centered design, interface design, online help systems, customization, end-user programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
