Been quiet but not idle… UXPA Talk - "DYNAMIC SINSIGN" 21st May

Although the blog has been quiet I haven't. I've been busy with my students, proofing my new book and working hard on my PhD. Some of the results of that work will appear on this blog over the coming weeks and presented to the members of User Experience Professionals Association in Edinburgh on the 21st May.

The talk is called Dynamic sinsign: Sign-action communicating experiential themes. Check out the link here to book a place.
http://uxpa-scotland.org/events.php
The talk is free for UXPA members, £10 for non-members, and £5 for students. Pay at the door. [Payment is a UXPA thing - I'm speaking for no fee so I have nothing to do with the money!]  

Venue
Skyscanner: Quartermile One, 15 Lauriston Pl, Edinburgh EH3 9EN. LATECOMERS CAN NOT BE ADMITTED  

Information

DYNAMIC SINSIGN:SIGN-ACTION COMMUNICATING EXPERIENTIAL THEMES

About the event
This talk emerges out of Dave’s current research-in-progress into interaction design from a Visual Communication perspective. Through a clever synthesis of Peircean semiotics and Martin Heidegger, Dave will discuss how the user experience can be revealed in a new way that connects HCI and interaction design with Visual Communication. The 45 minute illustrated talk is for any UX designer who’d like to get into the mind of the user to see what they experience in a hermeneutically direct way that personas and mental models can’t communicate.

In the first section of Dave’s talk he’ll set out the context for how Visual Communication can inform user experience:

• Visual Communication as a facilitator in behavioural change 
• Interpreting the Hidden 
• Signified Inspirational Data 

In the second section, Dave will illustrate his proposition with some early research experimental graphic outcomes and methods:

• Experience probes - capturing experiences 
• Experiential themes and dynamic sinsigns - visual interpretation 
• Visual Hermeneutic Circle - analysing and reduction 

Dave will reveal some of the philosophy that structures the central idea and implications. He will end by discussing the development of an experimental qualitative methodology to explore this further. The nascent Visual Phenomenological Methodology will be developed over the next year as part of his PhD, and Dave would welcome conversations with industrial partners to develop a Knowledge Transfer Partnership.

More about the speaker 
Dave is author of Interface Design: An introduction to visual communication in UI design to be published by Bloomsbury (AVA) in October 2013. Originally an illustrator, in 1997 he made the transition into interaction design designing interfaces. In the last decade he has become more interested in the user experience and the re-positioning of the design discipline of Visual Communication over the design of future interactions. His interest now lies in understanding the user's experience in the interaction and exploring how this can be communicated to other designers.

He is a lecturer in digital design and design researcher at Glasgow Caledonian University, teaching on BA(Hons) Graphic Design for Digital Media. He teaches across all 4 years, and in the 3rd year he teaches a user experience module called Design and the User to design students.

A member of the Interaction Design Association, he has academically published his research and is currently in the practical phase of a practice-based Visual Communication PhD at Edinburgh College of Art. Check out his academic profile http://gcal.academia.edu/DaveWood/ and blog http://internalexternal-2010.blogspot.co.uk/.

The Semiotics of Embodied Interaction… Connection Through Pragmatism

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

 

“we live in a world that has been vastly altered by our cognitive abilities such that we inhabit not only the empirical world of physical entities but also the world of sign systems, which are a direct result of our cumulative interactions with the world (and each other) over time. Thus, the relationship between the subject and the object is dealt in a pragmatic way, where external phenomena are experienced as signs that are meaningful to the organism and there is no separation of the two.” (p144)

Annotation

The experience of an empirical authentic relationship within an environment between the person and the objects in that environment is mediated by the inauthentic semiotics of what those same objects afford to the person as to how they can be used. O’Neill argues that there is no separation between the modes of a meaningful existence, “Because we perceive before we conceive, we find that the body is at the root of our conceptual apparatus as well as being able to engage with the world without having to think about it” (p158). As the human is an essential organism in the environment and not distinct from the environment. Seen in a pragmatic way, the semiotic signs from the objects in the environment communicates meanings that lead to action that impacts on the nature of the same environment.

The Semiotics of Embodied Interaction… Ready-to-hand and Present-at-hand Modes

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“Interacting with the media of all kinds then falls between these two modes of being. On one hand we perceive and encounter our media-rich environment directly, we manipulate it and transform it through our-ready-to-hand mode of being. On the other hand, we are constantly viewing, reading and interpreting mediated information through the reflective mode of encountering it as present-at-hand. At the same time we move between these two modes of being as we inscribe, interpret, transcribe and transform our mediated environment. In a very clear sense we are deeply entwined physically with the media in our environment and in another we are constantly making, and making sense of, the inscriptions that the media environment affords us.” (p140)

Annotation

An experience fluctuates between a ready-to-hand mode of being and a present-at-hand mode. The former involves the perception, interpretation, action and physically embodied manipulation using what is at hand in the experience of interacting within the present environment the person finds themselves in. The latter a more reflexive interpreting state of scanning what is available, making sense of the semiotic messages inscribed within the present environment the person finds themselves in.

The Semiotics of Embodied Interaction… The Phenomenology of Signification

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“From the Peircean perspective, the focus is on the phenomenology of signification, i.e. signs as they are experienced. This is very useful as a starting point for understanding how users might experience interactive media signs.” (p81)

Annotation

Drawing on work by Peirce, O’Neill conjures up a term that connects my research areas together. The phenomenology of signification looks at how semiotic signs are experienced. Although no more than a term at present, like aesthetics of interaction it presents a way of synthesising pragmatics, semiotics and phenomenology together to explore how visual communication can influence interaction design.

The Semiotics of Embodied Interaction… Horizon for Understanding

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“For Umberto Eco, for example, meaning is no longer an individual construct, as in the arbitrary semiotics of Saussure. It is now seen as the result of a process in which an individual takes part in society through the coding and decoding of his/her relationship with the cultural values and societal norms of the time.” (p72)

Annotation

Umberto Eco’s semiotics is far more embodied with an active relationship for a person, in their background culture and society, in interpreting the coding and de-coding of the norms for understanding. This makes any meaning arrived at not as an individual construct but one that is constructed from both the individual’s internal pre-understanding and external socio-cultural factors that form a horizon for the understanding. For Palmer (1969) the pre-understanding of the individual and the horizonality of the context in which the individual is interpreting the meaning dictate the terms in which any meaning is constructed.

The Semiotics of Embodied Interaction… False Affordances

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“Gaver goes on to clarify his version of affordance by distinguishing between affordances themselves (i.e. the properties of things) and information about them (i.e. what we perceive). Gaver does this in order to show how we can be mistaken about affordances, claiming that if there is no information available for an affordance then it is hidden and that if information suggests a non-existent affordance, a false affordance exists.” (p52)

Annotation
Gaver adds another perspective to affordance of false affordances. Without going to far down this argument what Gaver does provide here are clues to how the semiotics of visually communicating the affordance can be problematic. 

The Semiotics of Embodied Interaction… Affordances

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“Norman separates ‘real’ affordances (the physical properties of the world) from ‘perceived’ affordances (subjective representations in the mind), and is more concerned with the perceptual properties of affordances rather than the actual properties of real objects themselves. (…) The strength of Norman’s version of how perception and affordance work is that it attempts to solve the problem of how to explain the role of knowledge in understanding the world around us. However, the problem with Norman’s version of affordance is that it abandons the unique contribution of Gibson’s ideas in bridging the gap between the object and the subject.” (p51)

Annotation

Prof. Don Norman classifies different forms of affordances into those that are properties of the physical world (real), and those that are in the mind and are representative and subjective (perceived). It is the perceived affordances that Norman is more interested in and in doing so he distances himself away from Gibson’s original affordance concept. Lately Norman has refined his ideas on affordances now, unfortunately, referring to them as signifiers. In doing so he has moved affordances closer to semiotics but also now muddied the clear separations between the two. Affordances are about communicating potential action while semiotics defines how the visual communication will communicate the call to action. An example could be a water tap. An affordance would communicate how to get water from the tap. Whether the water is hot or cold is communicated through the semiotic signs of red or blue. The outcome of the semiotics isn’t the pouring water, but the communication of the expected temperature. The outcome of the affordance is how the tap is operated to access the pouring water. Both help in the ultimate outcome of accessing water but perform different tasks.

The Semiotics of Embodied Interaction… Perceptual Changes

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“[Gibson’s] Theory of Ecological Perception begins by considering the relationship between living entities that perceive things in the world and the environment that surrounds them. (…) Invariant information (…) specifies the persistence of the environment and of oneself. (…) The perceiver is then aware of her own existence in a persistent environment relative to her own and others’ movements within the environment. This awareness provides us with information about the possibilities for action that the world around us directly affords. (…) Affordances then (…) are an emergent property of the physical relationship between environment and the direct perceptual acts of embodied beings.” (p50)

Annotation

The relationship between ourselves in the world, and the environment that surrounds us at any one time, is an important perceptual one. The environment is always perceptively relative to the person in that environment. They are aware that if they move, spatially and temporally that environment persistently envelope them. The perceptual changes within that environment are what guide and suggest options. How these guides and options within an environment communicate to the person is described as affordances, “an emergent property of the physical relationship between environment and the direct perceptual acts of embodied beings.” (O’Neill, 2008, p50). The calls to action that are afforded are defined within the semiotics of the visual communication.

The Semiotics of Embodied Interaction… Semiotics and Phenomenology (and) Pragmatism

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.



“The cognitive theories from psychology, which tend to understand human beings as information processing units and phenomenological approaches, which turn towards the importance of the body and action without thought. An alternative strand of theory that is particularly relevant to our concerns with new media is that of semiotics, which tries to understand interaction from the perspective of signification and communication.” (p39)

Annotation
Here O’Neill begins to set out how semiotics is associated with a phenomenological (and even pragmatist) approach to understanding experience. Just as Ihde created a postphenomenology from a fusion of pragmatism and phenomenology, for me to create a visual communication phenomenological methodology the synthesising semiotic theory into understanding interactive experiences is crucial. 

The Semiotics of Embodied Interaction… Phenomenological Relationships

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“Heidegger makes explicit the idea that the nature of our being-in-the-world is not only related to the world we inhabit, but to the fact that we are aware that we exist in that world. In short, a fundamental aspect of our being-in-the-world is that we are concerned about (aware of) ourselves being-in-the-world. Furthermore, another aspect of Heidegger’s conception of ‘Being’ is that our being-in-the-world is a being-with-other-beings (being as people, things or entities that exist in the world around us). Those other beings may or may not have a concern for being with us, but we most definitely have a concern for being-with them as they have an effect on how we exist. Our consciousness and our knowledge of the world are deeply entwined with our phenomenological relationship to it.” (pp34-35)

Annotation
O’Neil in building his case for the semiotics of embodied interaction naturally takes a Heideggerian perspective seeing “our consciousness and our knowledge of the world [as being] deeply entwined with our phenomenological relationship to it” (p35). Our interconnectivity with others shapes how we see and interact with the world we perceive as our reality. 
 

Representational Theory… Inauthentic Mediated World

KENNEY, K. (2005) Representational Theory. In: K. SMITH, S. MORIARTY, G. BARBATSIS and K. KENNEY (Eds) Handbook of Visual Communication: Theory, Methods, and Media. New Jersey: Lawrence Erlbaum Associates. pp99-116.

“Behaviourists attempt to observe and measure the real world directly. Phenomenologists are exclusively interested in a person’s introspective experience. Semioticians and rhetoricians try to understand the linkages between our internal world and the external world, and that linkage is necessary, they believe, because the external world is always mediated by our senses and our mind. Whereas rhetoricians have investigated how humans create and manipulate symbols in order to persuade other humans, semioticians have been more interested in how humans (and other animals) interpret all kinds of signs, including symbols, that were created by other people, as well as natural signs, including symbols, that were created by other people, as well as natural signs that may have resulted from plants, animals, or inorganic matter. Both rhetoricians and semioticians, therefore, are concerned with how ‘signs’ mediate’ between the external world and our internal ‘world,’ or how a sign ‘stands for’ or ‘takes the place of’ something from the real world in the mind of a person.” (p99)

Annotation:

We exist in a mediated world that in the digital age is sometimes a re-mediated world. This existence can be framed as inauthentic (a Heideggerian concept). How someone navigates and orientates their way through this mediation, making sense of and interacting in their world is important to understand for interaction design. There are a variety of qualitative methods to observe and reveal this processing. Kenney discusses what aspects Behaviourists, Phenomenologists, Semioticians, and Rhetoricians are interested in studying, and where they overlap or conflict with each other.

The main grouping between semiotics and rhetoric focus on the relationship within the internal and external world of the processing of semiotic signs, how the message is (re)mediated, perceived and understood. Behaviourists obviously are interested in the exhibited behaviour within the world, of the manifestation of how this process affects them. Whereas the phenomenologist is interested in the internal processes of how they experience the perception, understanding and reaction to the message that changes their behaviour.

Visual communication in this context is concerned with both the internal and external processing of the semiotic-facilitated meaning. It is concerned with the shaping of the semiotics externally in the design, but can also be used to reveal and visualise the phenomenological internal processes of the experience, to “convert the sheer impact of the world into pictures of simulacra” (Harman, 2007, p#)

Creative Visualization… Ambiguity, Connotation & Denotation

DAKE, D. (2005b) Creative Visualization. In: K. SMITH, S. MORIARTY, G. BARBATSIS and K. KENNEY (Eds) Handbook of Visual Communication: Theory, Methods, and Media. New Jersey: Lawrence Erlbaum Associates. pp23-42.

“Can we trust a message that can be so differently interpreted? Ambiguity provides a vital function in aesthetic visual communication. Ambiguity slows the mind and suggests experiencing the image more fully in many dimensions. An idea is a temporary mental event; thus, words cannot always adequately convey the complex, structural, relational, an contextual nature of interrelationships among aesthetic phenomena.” (pp31-32)

Annotation:
Ambiguity should be a device used only when connotational meaning is required such as layered contextual meanings for adults and children. When only a denotational meaning is required ambiguity should be avoided. This returns the conversation back to a semiotic one to help construct an aesthetic that achieves its intended outcome that needed to be communicated. 

Aesthetics Theory… Visual logic

DAKE, D. (2005a) Aesthetics Theory. In: K. SMITH, S. MORIARTY, G. BARBATSIS and K. KENNEY (Eds) Handbook of Visual Communication: Theory, Methods, and Media. New Jersey: Lawrence Erlbaum Associates. pp3-22.

“Visual logic is defined here as a system of visual relationships that encourages a developed internal sense of image cohesiveness, integrity, and elegance necessary to attract attention and guide the viewer to a sense of aesthetic completion and comprehension. Six individual, interlocking principles guide the development of this sense of visual logic. 

These six principles are 
(a) Ambiguity and Meaning, (b) Control of Direction, (c) Ecological Relationships, (d) Tensional, (e) Unity, and (f) Realism.

(a) Ambiguity and Meaning

(…) The ambiguity and meaning principle suggests that, if aesthetic form and content are effectively related, the visually literate viewer is able to extract relevant information by concentrating on visible relationships and the nature of meaning-making in the human brain. Multiple meanings emerge later from the layered relationships with the other five principles.

(b) Control of Direction

(…) Individual viewers will arrive at some degree of individual connection and relationship to the image based on their individual wants, needs, and expectations. The individual image-maker can control many aspects of ‘how’ the viewer’s perceptual system interacts with visible relationships but not the final interpretation. The maker can direct the viewer’s eye throughout the image’s significant areas through directional clues, groupings of elements, and tensions. The maker can also choose to emphasize certain aspects of the image and to subordinate others in order to direct the viewer’s mind to significant matters. However, the sheer number of visible relationships will quickly overwhelm the parsing and decision-making functions of the brain. (…)

(c) Ecological Relationships

When interpreting the inherent visual logic imbedded in an image, the maker (during the act of ideation) and the viewer (in a process of perception) both become involved in an intimate reciprocal and ecological [The study of living things, their environment, and the relation between the two.] relationship with the image. The visual structure offers parameters for perception of possible multiple meaning levels, as well as the potential for fixation on one particular meaning or focused aesthetic response. The maker, through trial and error, shapes this image, and the viewer must empathize with this relationship while at the same time understanding his or her ecological interactions with the structure. (…)

(d) Tensional

(…) the visual elements on the picture plane’s flat surface have tensional relationships with each other, with sides of image, and with its center of the image. The sum total of all tensions, both explicitly perceived and implicitly apprehended, creates an aesthetic impression of mood and a visible foundation for the communicated message. (…)

(e) Unity

(…) Without unity, the experience for the viewer lacks cohesion, making communicative interpretation less sure. With unity, the viewer’s own natural perceptual abilities can interpret visual clues to determine the nature and type of visual message and its relationship to reality.

(f) Realism

Application of the realism principle provides a ground of information that is either believable and true or false and manipulative. Informed visual communication must consider the medium of realism selected by the message designer and its aesthetic and affective dimensions created within the viewer.

” (pp16-18)

Annotation:

The six principles of a Visual Logic mirror a semiotic structure built on perception, interpretation and meaning. The six principles of (a) ambiguity and meaning, (b) control of direction, (c) ecological relationships, (d) tensional, (e) unity, and (f) realism are complimentary to the semiotic process.

A semiotic meaning-journey follows this path Sender > Intention > Message > Transmission > Noise > Receiver > Destination. Principle (b) control of direction sits comfortably within the area that the Sender occupies. The designer is the sender, and as creator they can “direct the viewer’s eye throughout the [design’s] significant areas through directional clues [visual affordances], groupings of elements [visual hierarchy], and tensions [white space, layout, typography, image and colour].”

The Intention, the Message itself and the Transmission of it affect and are affected by the principles of (a) ambiguity and meaning and (c) ecological relationships. The designer’s intention reflected in the design is to facilitate the transmission of the message to the receiver (user). The “visible relationships” that a visual hierarchy communicates helps the user to “extract relevant information” from interpreting the meaning of these relationships. This process of interpretation is a “intimate reciprocal and ecological relationship” with the semiotics in the design visually communicating the intended message(s) at that particular moment in the interaction with the design.

The principles of (d) tensional and (e) unity clearly affect the Transmission of the Message and contribute to the Noise that can prevent the understanding and interpretation of the message. Principle (a) ambiguity and meaning also can become noise if the aesthetic form is conflicted. If the message(s) being communicated semiotically through the visual communication of the interaction are not cohesive and understandable to the user; or if the “natural perceptual abilities [of the user cannot interpret the] visual clues to determine the nature and type” of the visual message then miscommunication will occur. The noise that prevents the communication is a lack of unity between the message, the chosen semiotic sign, and the transmission of it. This noise could be attributed to the design choices in the layout and visual mood of the design. These elements are what are referred to as tensional in the six principles of Visual Logic.

Finally the Destination and Receiver are affected by the principles (e) unity and (f) realism. The level of or lack of unity in the semiotics of the design obviously has a direct impact on the user (receiver) once the design outcome has reached its destination (being used). The realism in the visual communication rests with the perception of the user as to whether they trust that the design will let them achieve whatever outcome they require (contextual to the genre the design operates in), such as does a corporate design reflect the business? does the design of an eCommerce website communicate trust?

Aesthetics Theory… Facilitate the Transmission

DAKE, D. (2005a) Aesthetics Theory. In: K. SMITH, S. MORIARTY, G. BARBATSIS and K. KENNEY (Eds) Handbook of Visual Communication: Theory, Methods, and Media. New Jersey: Lawrence Erlbaum Associates. pp3-22.

“A verbal analysis of aesthetics only deals with one part at a time and is therefore always incomplete in description and understanding. In the creative design of visual messages, as well as in creative viewing and interpretation, flexibility of structural construction is vital. Flexibility provides for the purely perceptual apprehension of novelty, originality, and message integrity. Ritualized, stereotyped, and repetitive messages are not as likely to attract or sustain viewer attention. Therefore, the message’s flexibility is crucial for discovering and imparting significant aesthetic aspects of meaning.” (p8)

Annotation:
The flexibility referred to here is referring to an aesthetic that is the apotheosis to the ‘visual design’ attitude. The richness that visual communication can bring rests in the ability to ensure that the design all works towards its intended outcome. The semiotic structure will provide a lexicon of visual language to facilitate the transmission of the intended collective message with the minimal disruption by noise. How this is achieved should not be prescriptive and not just utilitarian, nor should the design be homogenised into the lowest creative denominator for the intended audience. A successful aesthetic attracts and retains attention and focus. A poor aesthetic reduces the attention and length of time needed to perceive and interpret what actions need to be taken. There is a difference between a simplistic (and therefore a weak design), and a design that aids interaction because it makes interaction simple by easily facilitating its complexity through clarity of design.