The Semiotics of Embodied Interaction… From Concealed Potential to Visual Targets of our Conscious Use

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“As Michael Wheeler points out, most expertly, there are not just simply two ways of interacting with the world that surrounds us. These two poles provided by Heidegger, represent the extreme ends of a spectrum of interaction opportunities that blend into one another (…) where we are often shifting quickly from one mode to the other in a very inexpert ‘unready to hand’ way. What is perhaps most important about this approach to understanding our relationship to things in the world and interactive technologies in particular is that, while it does not deny that we might have some form of representational knowledge about the world, it focuses on our connection to the world through our bodies in the first instance, promoting the idea that we are so connected to our surroundings that we need not build mental models of the world around us in order to act, but that we might simply act through a ‘direct’ relationship to them. This is very different from the original cognitive approach to HCI” (p36)

Annotation
Ready-to-hand and present–at-hand interactive opportunities are opposite ends of ways of interacting within the world. They present an interesting directness to interaction, as they suggest that our relationship to the objects we use changes from concealed potential to visual targets of our conscious use. How this is approached can bypass tacit knowledge leading to an embodied response. As understanding is not always articulated through language it can be an embodiment leading to direct action. This direct action can be articulated and expressed through an interpreted meaning being derived from the calls to action that prompted the interaction.

The Semiotics of Embodied Interaction… Ready-to-Hand and Present–at-Hand

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“Heidegger essentially posits two different ways of being-in-the-world in relation to stuff or equipment, to use Heidegger’s term, that one finds there (physical entities, such as tools and technologies that for our purposes, we shall identify as media); these are ready-to-hand and present–at-hand. (…) Heidegger claims that our everyday encounters with the phenomena of our world are the ‘first’ way in which we come to understand them. (…) Through this process of interaction we develop skilful use of the material of the world and we in turn develop tacit, embodied knowledge or ‘know-how’ that allows us to cope smoothly with the world around us, enabling our immediate survival. (…) Present-at-hand is an essentially different ‘disclosure’ of being, whereby we are no longer engaged in using the equipment of the world, but we instead are thinking about it. Our activities are internal and mental rather physical and active. This type of being-in-the-world provides us with a second kind of knowledge, ‘know-that’ rather than ‘know-how’.” (p35)

Annotation
When discussing Heideggerian phenomenology in the context of interaction design then there are two ways to frame how we approach an interactive artefact: it may be either ready-to-hand or present–at-hand. The former is focused more on a physically active state of imminent use, but in the latter our activities are internal and cognitive, thinking about when to engage in active use. In both conditions we are aware of the interactive artefact without actually using it, but once we are ready we can begin interacting from these polar positions. 

The Semiotics of Embodied Interaction… Phenomenological Relationships

O’Neill, S. (2008) Interactive Media: The Semiotics of Embodied Interaction. London: Springer-Verlag.

“Heidegger makes explicit the idea that the nature of our being-in-the-world is not only related to the world we inhabit, but to the fact that we are aware that we exist in that world. In short, a fundamental aspect of our being-in-the-world is that we are concerned about (aware of) ourselves being-in-the-world. Furthermore, another aspect of Heidegger’s conception of ‘Being’ is that our being-in-the-world is a being-with-other-beings (being as people, things or entities that exist in the world around us). Those other beings may or may not have a concern for being with us, but we most definitely have a concern for being-with them as they have an effect on how we exist. Our consciousness and our knowledge of the world are deeply entwined with our phenomenological relationship to it.” (pp34-35)

Annotation
O’Neil in building his case for the semiotics of embodied interaction naturally takes a Heideggerian perspective seeing “our consciousness and our knowledge of the world [as being] deeply entwined with our phenomenological relationship to it” (p35). Our interconnectivity with others shapes how we see and interact with the world we perceive as our reality. 
 

Beneath Interpretation… What is Articulated?

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“A criterion for having an interpretation of some utterance or event would be an ability to express in some explicit, articulated form what that interpretation is. To interpret a text is thus to produce a text. Understanding, on the other hand, does not require linguistic articulation. A proper reaction, a shudder or tingle, may be enough to indicate one has understood. Some of the things we experience and understand are never captured by language, not only because their particular feel defies adequate linguistic expression but because we are not even aware of them as ‘things’ to describe. They are the felt background we presuppose when we start to articulate or to interpret.” (p127)

Annotation
Understanding is not always articulated through language but can be embodied in feeling within our senses. Therefore it does not require expression. In comparison the act of interpretation is active and produces an outcome that can be articulated and expressed.

Beneath Interpretation… As Problem-solving

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“While understanding is frequently a matter of smoothly coordinated, unproblematic handling of what we encounter, interpretation characteristically involves a problem-situation. We only stop to interpret in order to resolve a problem – some obscurity, ambiguity, contradiction, or, more recently, the professional academic problem of generating an interpretative problem. The intrinsic problem-solving character of interpretation explains why it involves conscious, deliberate inquiry. Solving a problem demands thinking, seeing the obvious does not.” (126)

Annotation
Problem-solving demands thought while seeing the obvious in front of our eyes does not. Interpretation involves deliberate and conscious inquiry and is intrinsically predicated on problem-solving within a problem-situation. 

Beneath Interpretation… Danger of Absolutism

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“First, the distinction between understanding and interpretation (…) cannot be distinguished by epistemological reliability, where understanding implies univocal truth while interpretation connotes pluralistic error. Nonetheless, understanding and interpretation are epistemologically different in terms of their functional relations: understanding initially grounds and guides interpretation, while the latter explores, validates, or modifies that initial ground of meaning.” (p125)

Annotation
There is a danger of seeing interpretation and understanding in absolute terms distinguishing the former as pluralistic, relativist opinion-based, while the latter as a justified belief based on only one possible meaning of truth. Interpretation “explores, validates, or modifies” (p125) the understanding of an established meaning that is grounded in our pre-understanding.

Beneath Interpretation… When Interpretation Halts

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“We seek an interpretation because we are not satisfied with the understanding we already have – feeling it partial, obscure, shallow, fragmented, or simply dull – and we want to make it fuller or more adequate. Yet the superior interpretation sought must be guided by that prior inadequate understanding. We no longer feel the need to interpret further when the new, fuller understanding that interpretation has supplied is felt to be satisfactory.” (p123)

Annotation
To seek interpretation is to not be satisfied with our existing understanding, but this pre-understanding guides our seeking for new interpretations. Once the new understanding is satisfied by the new interpretation then the interpretive process halts. 

Beneath Interpretation… Distinction Between Understanding and Interpretation

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“we can only test our prior understanding by subsequent interpretation (…). Considerations of this sort have led Gadamer and other hermeneutic universalists to the radical claim that ‘all understanding is interpretation.’ But this claim, I have argued, is not only uncompelling but misleading in suggesting that we can never understand anything without interpreting it. For in many cases we are simply satisfied with our initial understanding and do not go on to interpret; there are always other and usually better things to do. Moreover, if we could never understand anything without interpreting it, how could we ever understand the interpretation itself? It, too, would have to be interpreted, and so would its interpretation, and so on ad infinitum. As Wittgenstein notes, ‘Any interpretation still hangs in the air along with what it interprets.’ Interpretation must ultimately depend on some prior understanding, some ‘way of grasping…which is not an interpretation.’ This is just a point of philosophical grammar about how these notions are related: understanding grounds and guides interpretation, while interpretation enlarges, validates, or corrects understanding. We must remember that the distinction is functional or relational, not ontological. The prior and grounding understanding ‘which is not an interpretation’ may have been the product of prior interpretations, though now it is immediately grasped. Moreover, it need be an explicitly formulated or conscious understanding, and the ground it provides in not an incorrigible ground.” (p122)

Annotation
Shusterman sees the distinction between understanding and interpretation not as ‘all understanding is interpretation,’ nor a metaphysical state of being (ontological), but as relational or functional. The relationship can be described as, “understanding grounds and guides interpretation, while interpretation enlarges, validates, or corrects understanding” (p122). As he points out if understanding is interpretation and we could never understand anything without an interpretation, then “how could we ever understand the interpretation itself?” (ibid.).  Hirsh (1976) agrees and urges that when the person interprets they first are trying to match what they sense with what they already know in order to first understand before and interpretation and explanation can begin.  Validation is a factor in interpretation and can be placed within systematic process of understanding > construction of meaning > interpretation > validation. When validation is achieved it is “only with respect to known hypotheses and known facts” (Hirsch, 1967, p170). Palmer (1969) points out that in itself understanding is a preliminary act of interpretation from which future interpretations are built. 

Beneath Interpretation… Understanding as a Base and a Guide

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“understanding provides interpretation not only with a meaning-giving contrast, but with a meaning-giving ground. It supplies something on which to base and guide our interpretations, and represents something by which we can distinguish between different levels or sequential acts of interpretation.” (p120)

Annotation

Understanding what is experienced gives a base and guidance to interpreting the meaning. This base and guide can provide interpretation with contrasts by which different levels or sequential acts of interpretation can be distinguished. Hermeneutically this is structured within the circle of interpretation by understanding the parts and the whole of the experience.

Beneath Interpretation… Implying a Conscious Processing

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“In short, I am arguing that though all understanding is selective, not all selective understanding is interpretive. If understanding’s selection is neither conscious nor deliberate but prereflective and immediate, we have no reason to regard that selection or the resultant understanding as interpretation, since interpretation standardly implies some deliberate or at least conscious thinking, whereas understanding does not. We can understand something without thinking about it at all; but to interpret something, we need to think about it.” (p114

Annotation
Shusterman is arguing, “all understanding is selective, not all selective understanding is interpretive” (p114) as since interpretation implies a conscious processing of information to arrive at a meaning, any prereflective and/or immediate understanding does not result in an interpretation. He concludes that, “We can understand something without thinking about it at all; but to interpret something, we need to think about it.” (p114) 

Beneath Interpretation… Motivating and Prejudicing Understanding

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“Any understanding involves the human element that prestructures understanding in terms (and in service) of our interests, drives, and needs, which significantly overlap but also diverge among different societies and individuals. Moreover, for Nietzsche, Gadamer, and the pragmatists the fact that understanding is always motivated and prejudiced by our needs and values is a very good thing; it is what allows us to thrive and survive so that we can understand anything at all.” (p112

Annotation
For pragmatists our needs and personal values always motivate and prejudice understanding. This human element (featuring our drives, needs and interests) terms and prestructures any understanding we have.

Beneath Interpretation… No Understanding is Foundational

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“Since our current hermeneutic turn derives in large part from the rejection of foundationalism, it is not surprising that the central arguments for hermeneutic universalism turn on rejecting foundationalist ideas of transparent fact, absolute and univocal truth, and mind-independent objectivity. For such ideas underwrite the possibility of attaining some perfect God’s-eye grasp of things as they really are, independent of how we differently perceive them, a seeing or understanding that is free from corrigibility and perspectival pluralities and prejudices that we willingly recognize as intrinsic to all interpretation. I think the universalists are right to reject such foundational understanding, but wrong to conclude from this that all understanding is interpretation. Their mistake, a grave but simple one, is to equate the nonfoundational with the interpretive. In other words, what the universalists are successfully arguing is that all understanding is nonfoundational; that is always corrigible, perspectival, and somehow prejudiced or prestructured; that no meaningful experience is passively neutral and disinterestedly nonselective. But since, in the traditional foundationalist framework, interpretation is contrasted and designated as the form of nonfoundational understanding, the inferior foster home of all corrigible, perspectival perception, it is easy to confuse the view that no understanding is foundational with the view that all understanding is interpretive. Yet this confusion of hermeneutic universalism betrays an unseemly residual bond to the foundationalist framework, in the assumption that what is not foundational must be interpretive.” (pp108-109)

Annotation

There are intrinsic prejudices within interpretation as we all perceive within our own reality, collectively mediated within the socio-cultural conditions we share. Hermeneutic interpretation rejects any foundationalist ideas that there is an absolute truth, for a pluralism of perspectival truth that can be corrected, rectified, or reformed with fresh data. But there is a danger in assuming that all understanding that does not have a foundational base has to be interpretive. Prof. Shusterman (1991) cautions that a pragmatic universalist perspective on understanding as nonfoundational should not confuse “the view that no understanding is foundational with the view that all understanding is interpretive” (pp108-109). Equating the interpretative with the nonfoundational has a danger of seeing all understanding as interpretation. Palmer (1969) does have the distance to see understanding as a preliminary act of interpretation, and Bohman (1991) from a holistic perspective acknowledges that a ‘correct’ functionalist or deterministic interpretation cannot explain how humans process and experience the world.

Beneath Interpretation… Pragmatism Meets Phenomenology

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“Interpretation is also practiced and theorized in terms of formal structure with the aim not so much of exposing hidden meanings but of connecting unconcealed features and surfaces so as to see and present the work as a well-related whole.” (p108)

Annotation
The formal structure of interpretation from a pragmatist perspective seeks to connect the parts with the whole experience being interpreted. It is not so much focused on simply revealing the hidden, it is more interested in the connections between the internal and external features of an experience to arrive at a full understanding of its meaning. This separates this from a phenomenological approach that is concerned with revealing the hidden. So pragmatism supports interpretation and synthesised with phenomenology can provide a structure to not only examine an experience through the aesthetic, but also to strengthen the validity of a phenomenological methodology against accusations of being unscientific. Although a full Ihdean postphenomenology is not what is being advocated, there are lessons in the synthesising the best of both philosophies into a practical visual communication methodology. 

Beneath Interpretation… Pragmatists and Interpretation

SHUSTERMAN, R. (1991) Beneath Interpretation. In: D.R. HILEY, J.F. BOHMAN, and R. SHUSTERMAN (Eds.) The Interpretive Turn. Cornell University Press. pp102-128

“Pragmatists, like Nietzcheans, insist on rejecting the very idea of any foundational, mind-independent, and permanently fixed reality that could be grasped or even sensibly thought of without the mediation of human structuring. Such structuring or shaping of perception is today typically considered to be interpretation, and so we find contemporary pragmatists like Stanley Fish repeatedly insisting that interpretation comprises all of our meaningful and intelligent human activity, that ‘interpretation is the only game in town.’ All perception and understanding must be interpretation, since ‘information only comes in an interpreted form.’ Thus, even in our most primitive and initial seeing of an object, ‘interpretation has already done its work.’” (pp103-104)

Annotation
Through a pragmatist perspective all perception and understanding arises out of a mediated human structuring into meaningful interpretation. Pragmatists reject a reality that is independent of the mind, foundational and permanently ‘fixed.’ Interpretation is the conduit through which we construct our reality.