The Semiotics of Embodied Interaction… Intentionality

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“The problem of inter-subjectivity is that, while we can each experience the world in an ontological sense, how can we know that each of us is experiencing the same things, given that we do not have direct access to each other’s thoughts and experiences. (…) Dourish posits that intentionality sets up the relationship between embodied action and meaning (Dourish, 2001, p138).” (p38)

Annotation:

Harman (2007) posits that phenomenology can re-establish itself “by expanding the concept of intentionality to the point where it covers the entirety of the things themselves, thereby freeing us from the growing staleness of the philosophy of human access” (p123). Dourish (2001) sees in the Husserlian distinction between act and matter that the manifested intentionality of acting over the matter, of interacting, holds a relationship between the meaning of the interaction and the embodied action of the interaction. O’Neill (2008) concisely reminds that phenomenology offers a philosophy and methodology to reveal what others see as much as possible considering problem of a metaphysical state of being only being existentially experienced.  

Guerilla Metaphysics… Intentionality

HARMAN, G. (2007) Guerilla Metaphysics: Phenomenology and the Carpentry of Things. Open Court Publishing Company.

“The way to revive phenomenology is not through external rituals of compliance with Husserl’s vocabulary, but by expanding the concept of intentionality to the point where it covers the entirety of the things themselves, thereby freeing us from the growing staleness of the philosophy of human access. (…) Instead of maintaining the usual focus on categorical intuition, so favoured by disciples of Heidegger, I propose that we examine the simple Husserlian distinction between act and matter.” (p23)

Annotation:
Harman (2007) posits that phenomenology can re-establish itself “by expanding the concept of intentionality to the point where it covers the entirety of the things themselves, thereby freeing us from the growing staleness of the philosophy of human access” (p123). Dourish (2001) sees in the Husserlian distinction between act and matter that the manifested intentionality of acting over the matter, of interacting, holds a relationship between the meaning of the interaction and the embodied action of the interaction. O’Neill (2008) concisely reminds that phenomenology offers a philosophy and methodology to reveal what others see as much as possible considering problem of a metaphysical state of being only being existentially experienced. 

The Phenomenological Perspective

The exploration of designing interactions within a matrix of the user’s embodied and situated personal understanding is a HCI paradigm shift “to recognizing a plurality of perspectives (…) taking into account but not adjudicating the varying and perhaps conflicting perspectives of users.” (Harrison, 2007, pp7-8). The move to a phenomenological paradigm within HCI allows for a profitable linkage with Visual Communication to support Interaction Design.

Phenomenology is both a philosophical movement and a research methodology. The former provides a theoretical framework that connects Dourish’s embodied interactions, Suchman’s situated actions, McCullough’s digital ground to a pragmatic understanding (Dewey, 1980)(Shusterman, 1992) of the aesthetics of experience. In phenomenological research there are two forms, descriptive (eidetic) and interpretive (hermeneutic). Descriptive phenomenology follows the philosophy of Edward Husserl, and hermeneutic phenomenology the philosophy of Martin Heidegger (Lopez, 2004, p727). Heidegger uses the term Dasein to describe existence in respect to our own understanding of being in the world. Our “Being grows out of the average understanding of Being in which we are always involved” (1993, p49). This is ontological as to understand Being is “itself a determination of Being” (p54). This existential understanding is a “constitution-of-Being of the being that exists”(p55) in the world, and phenomenology is a concept of method through which to study the phenomena of Being, an expressed maxim of “To the things themselves!” (p72). Phenomenology investigates the Being of Beings, on studying the ‘how’ and ‘what’ meaning of a phenomena, making known the structures of Being. This investigation is hermeneutic, and it is in hermeneutic phenomenology that proposes that all understanding is interpretive (Johnson, 2000, p143). This ontological investigation of bringing out the Being of Beings helps thematise that structure.

Phenomenological reduction of these themes of Being within a specific phenomena (e.g. an aesthetic experience) begins with the apprehension of Being, to the understanding of the uncovered themes of Being, returning back through interpreting these themes of the studied experience, to the apprehension of the Being. This is a Hermeneutic Circle of interpretation, reducing the uncovered themes of the studied phenomena to “uncover commonalities and differences” (Benner, 1994, p104) seen through the eyes of the individual, to illuminate “that would have been overlooked in a purely descriptive approach” (Lopez, 2004, p734).

This hermeneutic process is contextual to a situated, cultural and historical “meaning of being in the world” (Earle, 2010, p288) (Johnson, 2000, p144) and the interpretation is conditional on the temporality of Being (Heidegger, 1982, p17) between the apprehension of Being and the understanding of the uncovered themes of Being within a studied situation. This systematic movement within a hermeneutic circle of interpretation affords the interpreter to check for “incongruities, puzzles, and unifying repeated concerns” (Benner, 1994, p113), and leads the interpreter through a cycle of “understanding, interpretation, and critique” (p120) to “uncover naturally occurring concerns and meanings” (p112) to understand the phenomena as directly as possible - as directly as experienced. Other qualitative methods attempt to examine the peripheral limitations or delimitations that surround it. In a phenomenological study the researcher enters the research open to understand the phenomena fully through the eyes of the participant. This is to understand their behaviour within that specific experience of the phenomena.

The framework for such a phenomenological research methodology is taken from two recognised academics Moustakas (1994) and van Manen (1990). Moustakas has proposed an actual staged framework for a methodology that researchers have used. This framework can be adapted for both eidetic and hermeneutic research, but to adapt it for an interpretive phenomenological study van Manen’s guidance needs to be followed. This framework, influenced primarily from nursing literature, will be discussed later in the paper and provides an established research methodology in which to explore the aesthetic experience. In doing so Visual Communication can be repositioned away from being seen as the end “artifice” and brought back to the earlier conceptualising design stages. The methodology as used to date is a written study, where each stage of writing and rewriting within a hermeneutic circle reduces the themes to a composite synthesis of meaning. van Manen advocates the writing process and offers five ways to approach writing up of the findings: thematically, analytically, exemplificatively, exegetically, or existentially (p173). He also offers a proviso to choosing the best approach that opens the methodology up beyond a purely textual outcome. Whilst he suggests the researcher can choose a combination of the above, decided by the nature of the studied phenomena, he also practically (or pragmatically) opens this up potentially beyond the textual. He calls on the researcher to be “creative in finding approaches and procedures uniquely suited” (p163) to both a particular project and the individual researcher. This is another a profitable linkage that Visual Communication can use to support Interaction Design through connecting to HCI. A purely textual representation of a phenomenological study of an experience is only partially useful for inspiring an interaction designer understand a particular experience. Visual data is a valuable source for both understanding and inspiration, and according to Benner (1994) the “use of interpretive phenomenology for interpreting visual sources of data is not yet well developed, but visual data are central to many lines of inquiry amenable to interpretive phenomenology, particularly social practices, embodied skills, and the study of lived experience.” (p120). This was sixteen years ago and through a literature search this remains under-developed. I propose within this paper how a Visual Communication Phenomenological Methodology can be developed and using a practical pilot project how it can be visually conducted.

The Interaction Design Perspective

The development of the discipline of Interaction Design has been summarised earlier in this paper, but Moggridge from the beginning strove for giving “aesthetic pleasure as well as lasting satisfaction and enjoyment” (2007, p14) in the work that was to be designed. Interaction designers Jon Kolko and Dan Saffer reflect upon this. Saffer summarises the scope of his discipline by detailing what it was not,
“It wasn’t product design exactly, but they were definitely designing products. Nor was it communication [graphic] design, although they used that discipline’s tools as well. It wasn’t computer science either, although a lot of it had to do with computers and software” (2006, p3).

Kolko sees the discipline as designing a person’s physical and emotional dialogue with an interactive artefact or system, leading to both a satisfactory emotional experience and a satisfactory engagement with the content. Once used, the experience should result in a form of behavioural change that is positive, and enjoyable. In my intention to reposition Visual Communication’s influence earlier in the Interaction Design process, it is crucial also to look below the aesthetics of the surface, the visual interface or form, into what Dunne describes as the ‘aesthetics of use’ (1999). To explore and drill-down below the surface of an interaction, into what Nake describes as the ‘subface’ (2008) and Lim et al. term the ‘interaction design space’, an interaction designer needs to gain knowledge about the nature of the interaction; the intent, needs and desires of the user; and the material attributes that can be manipulated to iteratively develop the design. The materials interaction designers use differs from the materials used within Product or Graphic Design (Lim, 2007) and are more ‘flexible, ungraspable, and phenomenal’ (ibid. p245). It is in understanding these attributes of design and their manipulation that creates an interactive experience greater than the attributes used (ibid. p239).

But as Interaction Design’s materials are not tangible, non-qualitative critics have argued that it cannot formalise a design process that is procedurally executable or repeatable. Kolko sees a need for Interaction Design to position itself into a duality that “emphasizes the human side of technology.” (2010, pp11-13). This physical and emotional dialogue is synonymous with McCarthy and Wright’s (2004) underdeveloped ‘emotional-volitional’ component of the relationship between the human and the technology. They discuss users now becoming active consumers who are no longer passive in the relationship with technology, but through their imaginations and emotional attachment to their chosen technology. This quality of experience is a “felt and sensual quality” (p13) within a situated moment that is more than an instrumental exotelic experience. It is what Dewey (1980) describes as an aesthetic experience, a refined form of everyday experience that is satisfying and creative.

Dewey, a pragmatist philosopher, describes aesthetic experience as the “conversion of resistance and tensions, of excitations that in themselves are temptations to diversion, into a movement toward an inclusive and fulfilling close.” (p58) Csikszentimihalyi in his psychology research on FLOW lists eight major components to an aesthetic experience. He describes a phenomenology of enjoyment that can frame an autotelic experience that Dewey would describe as aesthetic. Csikszentimihalyi’s eight components that may all feature together (or in cases only one) are: a chance of completing a task; concentration on actions; action has clear goals; immediate feedback on actions; a deep but effortless involvement; a sense of control over own actions; concern for the self disappears (yet paradoxically the sense of self emerges stronger afterwards); and the sense of the duration of time is altered (p49).
McCarthy and Wright argue that it is the dynamic involving “cumulation, conservation, tension and anticipation” (p64) within a user that is always moving toward a fulfilment where the diversion away from natural obstacles, resistances and tensions are overcome leading toward an outcome that Dewey describes as a close that is both fulfilling and inclusive. This instrumentality of means-end is not exclusively functional but emotionally felt. It sits within a existential situation where the event is always becoming, conditional on the context and the temporality of the situation. The aesthetic experience emerges from the lived experience, where the self can be lost in the moment but can return, feeling nourished and contented. McCarthy and Wright’s perspective on the implications of a ‘emotional-volitional’ component goes beyond a subjective state to the “irreducible totality of people acting, sensing, thinking, feeling, and making meaning in a setting, including the perception and sensation of their own actions” (p85). This is to draw the distinctions between an intrinsic and extrinsic understanding that meaning fluctuates between autotelic and exotelic experience: for both its own sake and for exterior purpose. Dourish (2004) suggests that meaningful experiences are situated within, and shaped by a person’s immersion within the experience.

McCullough develops the argument that actions are shaped by their contexts and that opportunities for participation become available through cognitively encountering them in a situated way. The embodied interaction is resultant from the surrounding possibilities for participation, but the interaction emerges and is shaped peripherally from possible affordances rather than directly imposed choices. The experience is cognitively embodied, with the meaning of it changing through interaction with the system or artefact through its creation and manipulation (McCullough, 2005, p126). The situated action from where the understanding of the experience emerges is at once cognitive, temporal, physical and social (Anderson, 2003). This shifts the focus of an interaction away from the control of the designer and onto a user’s experience of the artefact or system they are using (Gajendar cited in Kolko, 2010, p120). This McCullough sees as a shift of design values “objects to experiences, from performance to appropriateness, from procedure to situation, and from behaviour to intent” (2005, p50). This experience framed within a ‘emotional-volitional’ perspective leads towards a phenomenological approach to understanding and interpreting it. Reframing an experience of an interactive artefact, system or service within such a framework allows for the experience to begin to be understood by the interaction designer as close as possible through the eyes of the user. In doing so new insights in how to design a better user experience can be made through interpretations on human’s real lived useage. In the next section of the literature review this new paradigm will be discussed.