The Semiotics of Embodied Interaction… Behaviour Change

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“The key to solving this fundamental problem [_{compatibility of Heidegger’s, Merleau-Ponty’s and Peirce’s theories}] is in understanding the relationship between perception and conception. In other words, in developing a theory that takes into account how veridical perceptual experiences of the ‘real’ world that are derived through direct perception become, stored, represented or re-perceived in our minds as knowledge. Furthermore this theory also has to take into account how this knowledge is fed back into the world as mediated representations that signify that same knowledge in our heads, allowing us to communicate and socially construct the everyday world of our reality.” (p133)

Annotation

Phenomenologically it is difficult to know for certain what one person sees is the same for others, but through a socio-cultural consensus meaning is agreed, attributed, and mediated through a visual grammar of signs. These signify meanings that are socially constructed and that can be successfully interpreted leading to a change in behaviour that the designer is trying to facilitate. How this behaviour change unfolds can be phenomenologically revealed, and also visually communicated.

Aesthetics Theory… Triadic Interrelationship

DAKE, D. (2005a) Aesthetics Theory. In: K. SMITH, S. MORIARTY, G. BARBATSIS and K. KENNEY (Eds) Handbook of Visual Communication: Theory, Methods, and Media. New Jersey: Lawrence Erlbaum Associates. pp3-22.

“Aesthetics permeates all interactions between these three [object:maker:viewer] components of visual communication. Relationships may not immediately reveal the exact intentions of the creator or help the viewer discover any potential hidden interpretations; but intentions, of both the maker and the viewer can alter everyone’s perspective on imagistic meaning. Knowledge of meaning can become clearer through a deliberate process of analysis and interpretation. Heightened awareness of one’s own mental imagery is the first step to assessing this deeper aesthetic aspect of visual communication.” (p6)

Annotation:
The triadic interrelationship of object:maker:viewer (p6) to the interpretation of the meaning according to Dake is framed in an idea as the creator (designer) as author. This is too restrictive for a phenomenological-focused inquiry into the relationships between visual communication, aesthetics of interaction, perception and interpretation. If this triadic is expanded to not think of the creator as author but instead as a semiotic sender, and the viewer (user) as the receiver of the message, then the interrelationship between the two places focus on how they perceive the ‘object’ as a transmission of that meaning. Within interaction design this interrelationship is not a simply one-way transaction as the nature of interaction involves a feedback loop that informs where the interaction goes. Therefore there simply is not just one semiotic message being interpreted, but a continual semiotic dialogue leading the user towards a conclusion to their interactive task. The behavioural change that the visual communication of the dialogue affords is resultant on active interpretation within the user as they perform the interactions.

Visual Communication and Performance

Jorge Frascara in his excellent book Communication Design argues for a fresh look on the function of Graphic Design. By looking at graphic designs, like illustration, as outcomes of Visual Communication, it makes it easier to escape the accusation that Graphic Design is just artifice.

Graphic designers, notes HCI expert Bruce Tognazzini, are limited to the interface’s ‘surface’ - how it looks and the design strategy behind communicating the content structure (2003). Gillian Crampton Smith, former school director of Interaction Design Institute Ivrea, sees graphic designers’ role as more involved in the interactive design process “designing what a package is and what it does, and then designing what it will be like” (Aymer, 2001). Crampton Smith, a former graphic designer within software development is well placed to acknowledge the role graphic designers can and do play in interface design. The visual language of interaction design is built upon design axioms inherited from graphic design, learnt over decades of designing for print.

Unfortunately there are misconceptions held by many professionals outside the discipline as to what a graphic designer does. They have been perceived as “decorationists, elitists or servants of the consumerist machine” (Laurel, 2003) and their work as “frivolous or shallow” (O'Reilly, 2004). Over the last century graphic design is so “deeply ingrained in the texture of daily life that it is taken for granted” (Crowley, (2004). It has become pervasive and transparent. Yet graphic design is actually serving its purpose - visually communicating a message or visually structuring the functions of an interface. In the next section I will be examining this issue.

Interaction design expert Brenda Laurel reflects that labels such as ‘dictators of style’, decorationists, elitists, ‘servants of the consumerist machine’ were unfair misconceptions of graphic designers’ profession, marginalizing their contributions despite the pervasiveness of them within society. Most of its practice is subjective, instinctive and implicit, alternating between the “consideration of objective information and intuitive leaps” (Frascara, 2006). Graphic design, when designed well, can “inspire a behavourial change” in its audiences (Forlizzi & Lebbon, 2006).

In Communication Design, Frascara develops this argument, strengthening the case for Visual Communication to be understood properly and not prejudicially. “Although some designs can become ornaments, historical documents or aesthetic paradigms – once they’ve accomplished their primary goal – visual communication design is not just about looks; it is fundamentally about performance.” (p12) He argues that a designer is designing an "an event, an act in which the public interacts with the design." This is an objective that makes it the "design of communicational situations" that impacts on "the knowledge, the attitudes, and the behaviour of people" (p13) and this happens after the communication has happened. The communicational event that is the design happens over time. The visual and aesthetic strength of the design are dimensions in which the communication is contained. This is how Visual Communication can now be seen, repositioned if you like, beyond the perceived 'aesthetics of surface' and into the 'aesthetics of use'.

Barnard in his book Graphic Design as Communication (2005) also comes to a similar conclusion that Visual Communication is about the performance of the communication. He writes, "the pleasure of the image, its entertainment value, is experienced at the same time as its persuasive function. Indeed, some would argue that the pleasure engendered is an integral part of its rhetorical power. Nor is it the case that an example of graphic design will perform only one of these functions. There can be no piece of graphic design that is only decorative, or only informative. It is the case that any and all examples of graphic design will perform more than one of these functions.” (pp16-17) The rhetorical nature of the work, its semiological transmission of a message is all there to shape the design's performance, and a primary conclusion of a behavioural change in the person receiving the design.

Sidenote: As Frascara discusses Visual Communication in the book I feel that the term used for the title causes confusion. Yes Graphic Design is about communication through the visual relationships of image/text, but the term Communication Design is a term that weakens this relationship.

References used:

AYMER, G. (2001) Norman Cooking. Create Online. 8. p38-40. BALDWIN, J. and ROBERTS, L. (2006). Visual Communication. London: AVA Publishing Ltd.

BARNARD, M. (2005) Graphic Design as Communication. Abingdon: Routledge.

FORLIZZI, J. and LEBBON, C. (2006). From Formalism to Social Significance in Communication Design. In: A. Bennett, (Ed.), Design Studies: Theory and Research in Graphic Design - A Reader. New York: Princeton Architectural Press, 51-63.

FRASCARA, J. (2004) Communication Design: Principles, Methods and Practice. New York: Allworth Press.

FRASCARA, J. (2006). Graphic Design: Fine Art or Social Science. In: A. Bennett, (Ed.), Design Studies: Theory and Research in Graphic Design - A Reader. New York: Princeton Architectural Press, 26-35.

LAUREL, B. Ed (2003). Design Research: Methods and Perspectives. London: MIT Press.

O’REILLY, J. (2004) Thinking with Images. In: R. POYNOR, ed. Communicate: Independent British Graphic Design since the Sixties. London: Laurence King Publishing Ltd., pp216-231.

TOGNAZZINI, B. (2003) It's Time We Got Respect [online]. [Accessed 2nd January 2009]. Available from: http://www.asktog.com/columns/057ItsTimeWeGotRespect.html)