1.1 An overview of the data

The data used as the basis of the current paper includes corpus narratives and dialogues (CFINSL), pre-structured monologue data (SLM), and motion capture dialogue data (MOCAP), all collected or processed at the Sign Language Centre in the University of Jyväskylä, Finland. The corpus narratives consist of altogether 12 retellings of two picture-books, The Snowman, and Frog, where are you?, by signers between the ages of 20 and 60, recorded in a dialogue setting with multiple cameras (e.g. Jantunen et al. 2016b; Puupponen et al. 2016; Puupponen 2018). The overall duration of the material is 45 minutes and 12 seconds and it has been annotated in ELAN for manual signs and sentences, syntactic structure, head movement types, and torso movement types (Jantunen et al. 2016b; Puupponen et al. 2016; Puupponen 2018). From the 12 narratives, the interplay between head and torso movements has been annotated and analyzed for a subset of 6 narratives, and the functions of these overlapping torso and head movements has been annotated for a subset of 4 narratives (for a detailed discussion, see Puupponen 2018). The corpus dialogue data consists of altogether 6 dialogues from the same signers as in the narratives. In these dialogues, the signers discuss their experiences and thoughts on events and other matters related to the deaf world. The dialogue data has been annotated for manual signs and sentences. The examples from the dialogue data used in the current paper have been identified by means of an overall visual examination of the data, and with the search functions available in ELAN.

From the other semi-structured monologues, the discussion in the current paper focuses on a 1 minute 8-second-long stretch of video material, recorded by the Finnish National Association for the Deaf, in which a native FinSL signer discusses the relationship between young deaf people and the activities of the traditional national deaf associations. The data (consisting also of computer-vision based motion analysis) has been annotated for manual signs and sentences, as well as for head movements and their functions (see Puupponen 2012). Finally, in relation to the dialogue motion capture recordings, the data consists of altogether 2 minutes 15 seconds of synchronized motion capture data and digital video material of two continuous dialogues in which two FinSL signers talk about their work, studies, and everyday language use. The data has been annotated for manual signs, head movement types and functions of different head movements, and the analysis included both a detailed description of the different forms of head movements on the basis of the quantitative motion capture data, as well as an analysis of the form-function pairing of different types of head movements (a detailed description of the data, annotation and analysis can be found in Puupponen et al. 2015).

2.1 Universal categories & semiosis

C.S. Peirce’s semiotic theory (e.g. 1894; 1903a) forms the basis of the semiotic framework for the discussion on signers’ head movements in the current paper. It deals with all kinds of signification, that is, the process of how things become signs for something else. According to Peirce, knowledge, and the world in general, are comprised of signs. Peirce’s conception of signification is based on so-called universal categories, a phenomenological triadic distinction that forms the basis of his philosophy (Peirce 1903b). According to Peirce, different aspects of reality can be described on the basis of how things exist experientially. The first of the categories, Firstness, is feeling itself: a raw experience of something without thought or conscious processing ([Quality of] Feeling). It is an experience that is not compared, proportioned or connected with any other issue; it is a sort of observation of a quality without a relation to another entity, for example an experience of color. (Peirce 1894; 1903b.) Secondness (which he also called Reaction or Struggle) is an element in which two things are in an influential connection with each other. It is a feeling that materializes in relation to another feeling, such as a reflex-like motion emerging because of a stimulus, or an eye-blink emerging in order to moisten one’s eye. According to Peirce, things in our experiences and consciousness come true as these types of relations, while single raw feelings exist only when we are not fully awake. (Peirce 1894; 1903b.)

The third category, or Thirdness, is thinking: a conscious process in which one learns that something is controlled or governed by a rule or a habit. Thirdness is a tripartite relation in which two things are connected through a third thing, which is a habit or a law. Signification is always an example of thirdness. Something stands for some other thing, and in so doing, conveys an idea of this other thing. For example, an eye-blink, the physical event caused by the necessity of keeping one’s eye moist and clean, becomes a sign when it co-occurs with the boundaries of signed discourse and therefore functions as one of the signals that help us perceive discourse structure. Only the elements of thirdness are intelligent, non-mechanical processes that include reasoning, signs and learning. (Peirce 1894.) Actually, according to Peirce, human cognition—thinking, learning, rationality—is in itself an act of signification (it comes true through signification) and therefore an example of the third universal category.

The core concept of Peirce’s signification is semiosis, the event involved in the interpretation of a sign. Semiosis is a tripartite relation which consists of the sign, the object, and the interpretant. In semiosis, things become signs because they are interpreted as standing for something else, their objects (Peirce 1903a).

“I define a sign as anything which is so determined by something else, called its Object, and so determines an effect upon a person, which effect I call its interpretant, that the latter is thereby mediately determined by the former.” (Peirce 1903a: 478.)

In Peirce’s semiosis anything may be a sign. A sign is a sign only through this process of interpretation, because of the fact that it ends up being interpreted as a sign of something. In semiosis, an entity or an event, such as a flash of lightning, becomes a sign when it is interpreted as standing for its object, a thunderstorm. As Kockelman (2005) points out, the social practice of joint attention is another example of this. In a situation in which a child turns their attention to what their parent is observing, the change in the child’s attention is the interpretant, the direction of the parent’s attention is the sign, and the thing that the parent is attending to is the object (Kockelman 2005). Nieminen (2010) argues that identifying different forms of a language is in itself already an interpretation of a sign in the Peircean sense, and noticing and distinguishing boundaries in the speech signal are examples of semiotic processes, even though these signs would or could not have a meaning in the Saussurean (1916) sense. In other words, signs may signify without being interpreted as meaningful. Some signs do not have a static, conventionalized way of being interpreted as standing for some precise event or entity. They do, however, connect to other signs to form intentional and meaningful social actions (Enfield 2009). The characteristics of a semiotic sign will be discussed in more detail in Section 2.2.

The second part of Peirce’s semiosis, the object, is what the sign stands for (Peirce 1894; 1903a; Kockelman 2005). As was pointed out above, an object may be something perceivable, such as something a person is directing their attention to (Peirce 1903a; Kockelman 2005). On the other hand, according to Kockelman (2005: 242), it may be something less precise and more abstract, such as something that directs us to a certain behavior (e.g. stroking a cat) once we have observed a sign (e.g. the cat’s purring sound). The purring stands for something, but that something is not as easy to determine as, for example, the object of a pointing action would be. The object may emerge as a synthesis of possible interpretations, such as seeing the purring of a cat as a sign for a purpose which the cat is trying to achieve (e.g. it wants the stroking to continue) (Kockelman 2005). Kockelman (2005) continues that objects may be types which emerge through different contexts of use (e.g. a proposition of an assertion) or they may be tokens in particular contexts of use (e.g. the state of affairs in a specific assertion). Peirce himself also distinguished between so-called immediate objects and dynamic objects. Peirce’s immediate object is an object that does not exist independently of the sign; it is the representational object of the sign in a certain sign event, that is, a token (Peirce 1906; Kockelman 2005). A dynamic object is independent of the sign and it determines whether the sign exists. It is not dependent on a particular sign event, and so it is a type rather than a token (Peirce 1906; Kockelman 2005). Type and token are important notions also in relation to the characteristics of a semiotic sign, which will be discussed further in Section 2.2.

The third part of the triadic Peircean semiosis, the interpretant, is what happens in our minds when we experience that a sign stands for its object (Peirce 1903a). It is what our mind does as we “read” or “translate” the sign, that is, when we realize that one thing stands for something else. (Peirce 1903a; Atkin 2013.) Kockelman (2005: 251) points out that the Peircean concept of interpretant does not have to be mental or verbal, instead it may be “embodied in actual behavior”. According to Kockelman, these embodied interpretants may be bodily processes without any physical effort or commitment. They are changes in bodily state, such as a feeling of pain or blushing. These interpretants are themselves often perceived as signs by the person experiencing them, or by someone else who is witnessing them (Kockelman 2005). For example, feeling pain while burning one’s finger on a hot stove is an embodied interpretant of the sign, the heat from the stove. This feeling of pain is at the same time an interpreter and a sign. The feeling is simultaneously an effect from a previous sign (the thing causing the pain) and a sign for the next event, a reflex-like hand motion away from the heat. The motion event is a new interpretant which includes action. And so on. According to Kockelman (2005: 239), “most objects and interpretants are themselves signs – and so the three-fold relationality continues indefinitely: every component of one third is simultaneously (and/or sequentially) a component of another third.” Other embodied interpretants may be energetic behavioral processes or representations. The former “involve physical or mental effort”; people cause them to happen, without necessarily involving “purpose, intention or planning”, like, for example, stretching one’s neck in order to see the source of a loud sound. The latter involve signs with propositional contents or thoughts, such as the concept underlying the word ‘cat’ (Kockelman 2005).

In Peirce’s semiosis, signs are first recognized as signs, after which the function or meaning of the sign is gradually defined in the process of semiosis, which moves towards its final interpretation. This process may include several stages, in which the object of the previous stage (the immediate object) emerges as a sign for the following stage of interpretation, and which, in the end, reaches the final stage (the dynamic object). However, this process may not be linear or straightforward and it need not reach a final conclusion. All signs do not have an inherent characteristic that ensures that they are interpreted in a certain way (Peirce 1903a; Kockelman 2005; Nieminen 2010).

2.2 Sign

Peirce’s classification of signs—in its many versions—may be the best known part of Peirce’s semiotic thinking. The discussion of head movements given in the current paper has its roots in Peirce’s first and interim typologies of signs, published in 1894 and 1903. The interim typology is often regarded as the most complete and coherent presentation of his classification of semiotic signs. (e.g. Atkin 2013). In this typology, signs are defined according to three dimensions, which are based on Peirce’s three universal categories of experience: (i) what signs are themselves, (ii) what signs are when viewing the connection between signs and their objects, and (iii) what signs are on the basis of their interpretant. (Peirce 1903a.) For the purposes of the current paper I will briefly present dimension (i), after which the discussion will focus on dimension (ii) which will be referred to as the ground of the sign and which is the basis of the discussion of signers’ head movements in Section 3 of this paper.

When describing dimension (i), Peirce (1903a) defined signs as qualisigns, sinsigns or legisigns. A qualisign is a quality, a characteristic that may perform as a sign. An example of a qualisign is the quality of a color, such as the property of ‘redness’. Actually, a qualisign is not really a sign, and it does not function as a sign in reality. Instead it is an abstract quality which is potentially a sign, such as an impression of a movement that might be a sign of a sign language. When a qualisign manifests and is interpreted as a sign, it always becomes a sinsign. A sinsign is a single actually existing entity or event which functions as a sign. Although a sinsign is not just a quality—like a qualisign—it is a sign because of its qualities, that is, because of its qualisigns. An example of a sinsign would be, for example, a color sample. Finally, a legisign is a law or a convention that functions as a sign. It is a general type that has been established as significant, and which manifests as replicas, tokens of that type. These replicas are sinsigns of the legisign. According to Peirce, single instances of a certain word in a text are all occurrences (i.e. sinsigns) of the same word (i.e. legisign). A replica is a sinsign which is significant because of the law that creates the legisign. It is not an exceptional incident which becomes significant for other reasons, as are other types of sinsigns. (Peirce 1903a; Parmentier 1994; Kockelman 2005; Nieminen 2010.)

Let us look at three different signs: a word that has been uttered, a national flag, and an empty bucket close to wild blueberry bushes in the forest. Of these, an uttered word and a national flag are based on a shared norm: the sign is connected to its object through a convention. A particular word such as ‘cat’ uttered at a given moment (i.e. a sinsign) is a token (i.e.replica) of a more general type (i.e. a legisign): the normative connection between any instance of the word and its shared meaning or function (Peirce 1903a; Kockelman 2005). However, all sinsigns are not replicas: they do not have a type. The bucket in the forest may become a sign when interpreted as indicating that someone is planning or has at some point planned to pick the berries nearby. It can be interpreted as a sign for someone not far away from the location of the bucket (‘someone is coming back soon to continue with this task’). On the other hand, it might result in the thought that the bucket has been left there by accident (‘someone has forgotten their bucket’) or in concern (‘I hope nothing unexpected and serious has happened to the owner of this bucket’). The interpretation of the sign depends on the situation and may vary according to the interpreter: there is no rule or norm for it to be interpreted in one particular way. It is a single token without a type. In addition, a bucket in itself, if put into another context, does not invoke the same interpretation; it does not necessarily result in any interpretation of signification. If it is interpreted as standing for something else, it is a singular event which functions as a sign (Peirce 1903a). Kockelman (2005) calls these types of signs singularities and goes on to say that most sign events are singularities, and that semiosis is actually an inductive, rather than a deductive process. That is, the meaning of a type is induced through a token, not the other way around; or the meaning of a token emerges from other co-occurring tokens, potentially abstracting to a type (Kockelman 2005).

2.3 Ground of the sign: Iconicity, indexicality and symbolicity

In relation to dimension (ii), Peirce (1903a) defines signs as icons, indices and symbols on the basis of the connection between the sign and its object, that is, the ground of the sign. An icon is a sign that represents its object through its qualities. An icon and its object share these qualities, that is, an icon resembles its object in some way. An example would be a good portrait of a person. (Peirce 1903a.) An index, on the other hand, connects to its object primarily through an actual connection, contiguity in time and space, such as an effect or an impact (Peirce 1903a; Kockelman 2005). According to Peirce, this connection does not induce an analogical association due to the similarities between the sign and its object, as with iconic signs. Instead, an index is physically connected to its object. This connection is only noticed; interpretation emerges when the physical connection is observed to be conventional (Peirce 1903a). Examples of indexical signs would be smoke (as an index of fire), the sound of rain, a footprint, a pheromone, or a weather-vane. The physical connection between a sign and its object is what makes an index a sign, although indexical signs may have iconic features as well (Peirce 1903a). For example, a sign such as the aforementioned weather-vane includes the association of the similarity of the direction of the object’s orientation and the direction of the wind (i.e. iconicity) (Parmentier 1994; Kockelman 2005). Finally, a symbol represents its object through a law-like relationship and is therefore connected to it by means of a convention. Examples of symbols are the lexical words or signs of a natural language. From this conventionality it follows that symbols denote that something is of a certain kind, rather than indicating a single particular thing (Peirce 1894). According to Peirce (1903a), symbols are always types, generalizations of single entities. This means that symbols are always legisigns which manifest as replicas, and the object of a symbol is more general than specific.

According to Peirce, icons, indices and symbols are essential for human cognition and reasoning (Peirce 1984). However, Peirce describes these different representative types as features rather than categories. Peirce (1984: 10) states that in reasoning we must always “use a mixture of likenesses, indices, and symbols”. Furthermore, iconic, indexical and symbolic features may, and usually do, coexist in single signs. Already in his early account of semiotic signs, Peirce (1894) points out that a single sign may simultaneously be an icon and an index, and that symbols often have an imitative origin. An icon does not necessarily have indexical or symbolic features although this is possible, whereas an index must have iconic features (i.e. information about or qualities corresponding to its object), and a symbol must have iconic and indexical features (i.e. directing attention and providing information concerning it or indexing a code) (Peirce 1903a; Silverstein 1955; Parmentier 1994; Kockelman 2005). This same inclusivity can be found in Peirce’s Universal Categories: Secondness includes Firstness, and Thirdness includes both Firstness and Secondness. As Kockelman (2005: 246) points out, because of these inclusive relations “it is best to talk about iconic, indexical, or symbolic grounds, rather than to talk about icons, indices, and symbols per se”.

Let us consider one frequent example of this connectedness, a footprint. One can easily see how a footprint in the sand has iconic, indexical as well as symbolic features. The indexicality of the footprint lies in its physical connection to its source: the weight of a person’s foot leaving a trace in the sand. The indexicality in the sign links an event, i.e. someone walking, to something in time and space, i.e. that the walking occurred in that particular location. In addition, the indexicality links the sign to the foot itself, to the person who left the trace behind. This type of spatial, temporal and causal contiguity forms the indexical ground of the sign (Peirce 1903a; Kockelman 2005; Enfield 2009). On the other hand, identifying the similarity between a trace on the ground and a characteristic of its source is based on iconicity: association between the qualities of the sign, i.e. the shape and size of the footprint, and the qualities of its object, i.e. the shape and size of a foot. This association is a key characteristic in what makes a footprint a footprint, in just the same way as is the physical phenomenon through which it exists. Finally, although we may see only parts of a footprint on muddy ground, on asphalt, or on a beach – left by a shoe, a bare foot, or high heels – we still might recognize what it is: we are guided by our knowledge (which is based on convention) of what footprints are, how they are formed and what they look like (and that they are often imperfect, so to speak). The different semiotic grounds of the sign come together when we use and interpret signs, and construct meaning.

In the following discussion on signers’ head movements, with iconicity I refer to a relation between the sign and its object which is based on perceptual analogies (see Peirce 1903a; Parmentier 1994; Kockelman 2005; Enfield 2009; Dingemanse et al. 2015). Indexicality is defined as a relation between the sign and its object that is based on spatial, temporal or causal contiguity (see Peirce 1903a; Parmentier 1994; Kockelman 2005; Enfield 2009). Symbolicity is referred to as a relation between the sign and its object that is based on a norm in a community according to which the sign is regarded as standing for the object (see Peirce 1903a; Kockelman 2005; Enfield 2009). All of these three types of signification are referred to as the grounds of a sign or semiotic strategies. The paper makes no distinction between these two terms. Conventions and automatization emerging in one person’s communicative actions are referred to as entrenchment, while socially shared conventions are referred to as conventionality (e.g. Wilcox & Xavier 2013).

2.4 Utterances as actions: Semiotic complexity and pragmatic unity

If we broaden our perspective from single semiotic signs to whole utterances, intentional social actions occurring in discourse, it is evident that they are semiotically complex, especially if one counts the signals from different sign mediums—such as speech, movements of the body etc.—as parts of one and the same utterance (e.g. Kendon 2004; Enfield 2009). According to Enfield (2009: 15), in a communicative action, signs from different mediums come together into a “communicative move that incorporates multiple signs of multiple types”, which he calls a composite utterance. Some of these signs are more conventional (e.g. words, emblems), some unconventional (e.g. pitch, depicting hand gestures), while others are hybrids of conventional and unconventional features (e.g. pointing actions). The interpretation of these holistic chunks of semiotic content happens through “recognition and bringing together of these multiple signs” while presuming “pragmatic unity despite semiotic complexity” (Enfield 2009: 11). That is, although utterances are semiotically complex and consist of signification done simultaneously and multimodally with different sign mediums, these different layers of signification are connected and come to exist together as parts of one event which is driven by intentional social action (Enfield 2009). While investigating signals such as head movements in signed interaction, it is important to acknowledge that their meanings and functions do not emerge, and are not interpreted in, isolation. They are a part of multimodal social actions of communication in which different semiotic resources are available.

In order to maintain clarity in the argumentation regarding the semiotic features of signers’ head movements and the semiotic versatility in signers’ communicative actions, I wish to make the following terminological and conceptual distinctions: physical medium, sensory modality, sign medium (i.e. modality of information), and semiotic dimension (applied from e.g. Kress and van Leeuwen 2001; Bernsen 2002; de Ruiter et al. 2003; Enfield 2009). With physical medium I refer to the medium of phenomena such as light, movement and space, which are relevant for our sensory systems and sensory, mental and social activities. With sensory modality I refer to sensory systems such as vision, hearing, touch and balance. The term sign medium will be used to refer to the actions of different body parts: the hands, face, head and torso.⁴ With sign medium Enfield (2009) refers to modalities of semiotic information such as speech, hand gesture, eye-gaze and so on. For the purposes of the current paper and its approach to language (see Section 1), gesture is not considered to form a coherent sign medium, and it is not differentiated from the “linguistic component” (cf. Enfield 2009: 13). Finally, with semiotic dimension I refer to those features in signals of a specific sign medium that can independently vary simultaneously with other dimensions (Kress and van Leeuwen 2001; Enfield 2009). These could be, for example, the loudness of sound (which cannot be loud and quiet simultaneously) or the amplitude of the displacement of a hand movement (which cannot be a long and a short distance simultaneously). Different parts of a signer’s body may produce a variety of movements and positions (i.e. potential signs in different sign mediums) using additional kinematic features available for signification, such as velocity and displacement amplitude (i.e. semiotic dimensions).

3.1 Iconic features in head movements

According to the typology in Figure 1, the iconic strategy is important in head movements which resemble their objects either through enactment or metaphorically (see Figure 1). When a head movement has an iconic relation to its object, its interpretation involves association of the similarities between some characteristics of the sign and some characteristics of its object. Enacting head movements, such as the one in Figure 2, mimically enact the head movements of a discourse referent. Figure 2 presents an example of an utterance in which the head movements represent the head movements of a character in a story (a boy). While in the beginning of the example it is debatable whether the head tilt enacts the actions of a referent, the subsequent sideways rotations of the head (i.e. head turns left and right) are clearly enacting movements depicting the actions of the referent (looking left and right). During the enacting head turns, the hands produce the indexical verbal sign LOOK, also directed to the left and right side of the signing space. The indexicality of manual signs is briefly discussed in Section 4.2 of the paper.

Figure 2

An example of pantomimic head movements (head turns). CFINSL_005_05_00:00:35.⁵

Enacting head movements are signs that connect to their objects most prominently through iconicity. They are a part of an embodied depiction of the thoughts, sayings or actions of discourse referents, in which the content is enacted from the referents’ point of view—a discourse strategy often referred to as constructed action (e.g. Hodge & Ferrara 2014; Cormier et al. 2015). In the enactment, the actions of referents are projected in a size and manner that resembles real life (e.g. Ferrara & Halvorsen 2017), which in the case of human referents equals the embodied, visual and haptic engagement of human beings in the world. This is in contrast to observer-viewpoint description with fully or partly lexical signs (Johnston & Schembri 2010; Johnston 2012), in which actions are presented as distanced, small-scale description without enactment (e.g. Ferrara & Halvorsen 2017; Ferrara & Hodge 2018). These two strategies may also overlap. As depiction involves actions of different parts of the signer’s body, the hands may produce descriptive contents from the observer-viewpoint while other parts of the body are involved in enactment, as is the case in the example in Figure 2. In some recent studies discussing the different degrees of enactment (Cormier et al. 2015; Jantunen et al. 2018a; b), utterances consisting of both observer-viewpoint description and enactment are referred to as reduced or subtle enactment, whereas overt enactment refers to enactments in which hands are involved in the mimic depiction. Furthermore, formally similar or identical manual actions may emerge in either observer-viewpoint description or enactment, depending on the situation (e.g. Johnston & Ferrara 2012). Finally, in enacting movements, the direction of the head movement or the orientation of the face may indicate the relations and semantic roles between discourse referents.

The second type of iconicity in actions of the head, metaphorical iconicity, emerges, for example, in those head movements in which the direction of the movement is associated with an abstract image of a sagittal time line (Figure 3). In this case, forward and backward directed movements of the head or the whole body refer to the linear organization of things in time. Content associated with the future, or with subsequent events in the organization of things (‘later’, ‘after’), is produced with movements directed forward, whereas meanings related to the present, the past or anterior (‘earlier’, ‘before’) are produced with movements directed backward. I suggest that in these movements there is an iconic connection between specific features of the sign, i.e. the direction of the movement, and its object. The object in this case is a metaphorical spatial conceptualization of time (see Cooperrider & Nunez 2009): a visual and embodied mental image, common in the Western world, according to which time is spatially mapped, with the ego as a reference point, so that the future is something ‘in front of us’ and the past ‘behind us’ (Cooperrider & Nunez 2009; see also Lakoff & Johnson 1980; Wilcox 2000; Taub 2001; Cienki & Müller 2008). These metaphorically iconic sagittal head movements are optional. They may emerge in utterances with deictic signs, such as the FinSL signs FUTURE, and NOW, with manually produced indexical time-lines, or the temporal organization of events may be expressed without any particular head movements. At present, we do not know how frequent they actually are in utterances with time references, and whether they can alone anchor manually signed content to conceptual time-lines, without manual indexicality.

Figure 3

An example of a metaphorically iconic head movement (head pull). CFINSL_005_06_00:03:56.

However, according to earlier research on head movements in FinSL (Puupponen et al. 2015), the connection between the direction of a head movement and the linear organization of things in time is not categorical. The surrounding discourse context is a relevant factor in how head movements are directed: the movements may be coincidental, caused by kinematic features of manual movements (Woll 2009; Puupponen et al. 2015), or the contrast between ‘future’ and ‘past’ or ‘later event’ and ‘earlier event’ may be emphasized with movements produced contrariwise (i.e. ‘future’ backwards, ‘past’ forwards). In this case it is indeed this contrastive function which is relevant in the movements: two or more things are placed in opposite, alternative, comparative, or corrective relations to each other (see Johnston 1992; Wilbur & Patschke 1999; van der Kooij & Crasborn 2006; Crasborn & van der Kooij 2013; Puupponen et al. 2015; Jantunen 2016; Lackner 2017). I suggest here that when the contrastive function of head movements is not connected to a metaphoric concept of a sagittal time-line, its source of iconicity changes (see section 3.4) and the indexical ground becomes increasingly relevant. Contrastive head movements include indexical features of organizing the discourse structure and creating a spatial relation between the contrasted manually signed contents, which is discussed further in Section 3.2.2. It should also be noted that the indexical placement for showing opposition, alternatives etc. may be considered to include iconicity, that is, analogical association (e.g. opposites visualized at opposite sides of the signing space). Finally, metaphoric time-lines in sign languages are not only sagittal: the organization of events in time may be visualized also horizontally or circularly, as has been noted also in the manual gesturing of speakers from several different cultural surroundings (for a summary, see Cooperrider & Nunez 2009). However, in the data from which the current paper draws, head movements have not been found to iconically represent these other time metaphors.

In sum, the iconic strategies in signs produced with the head are most prominent and crucial in enactment, although iconicity is more or less involved in the production and interpretation of other types of head movements as well (see section 3.4). Enacting movements vary in form according to the needs of the depiction: movements and positions of the head imitate their object, that is, the activity of the head of a discourse referent, rather freely. They are often one part of a holistic iconicity in which their meaning emerges only in relation to the actions of other parts of the body: the body, head, and face form a unified depiction that is interpreted as a whole, and the head movement alone cannot be seen as having an independent meaning (head movements and the holistic nature of signers’ actions are discussed in Section 4.1). However, enactment need not involve all the aforementioned sign mediums.

Non-enacting iconic head movements are, for example, movements visually representing a time metaphor. In this case the central imitating feature was found to be the direction of the movement. Together with the direction of the manual movements, the movements of the signers’ body embody and visualize the imagistic dimensions of human temporal reasoning. On the basis of preliminary examination of the data, it is suggested here that head movements which have non-enacting iconic features are less common than enacting movements. However, this observation is given as a hypothesis for further investigation as it is not based on frequency counts.

3.2 Indexical features in head movements

3.2.2 Indicating referents

According to the typology, indexical features are prominent in head movements that indicate discourse referents or indicate discourse structure. Movements that indicate discourse referents have been analyzed as signs that direct attention to their objects or bring objects to someone’s attention (see Clark 2003; Cooperrider & Nunez 2009). The first type of movement is pointing movements of the head referring to specific referents. These movements function as manual pointing: they project a vector from a specific body part to a direction, location or an entity (Kita 2003). They may identify—introduce, re-introduce and maintain—referents which are present at the moment of signing or they may be used to refer to imaginary referents, that is, metaphoric locations in the signing space in front of the signer to which referents have been placed earlier in the discourse (for an overview see e.g. Liddell 2003; Perniss & Özyürek 2015). This strategy has been referred to as abstract deixis in relation to co-speech gesturing (McNeill et al. 1993). When pointing with manual signs, eye-gaze and movements of the head are directed to these locations, they organize discourse with anaphoric references. With speakers, reference tracking with manual gestures has been found to occur more in the re-introduction of referents than while maintaining them (Gullberg 2006; Perniss & Özyürek 2015). It is unclear whether pointing with the head and eye gaze alone may be enough when an imaginary referent is established for the first time in the discourse.

Pointing head movements may emerge together with indexical manual signs or independently, without manual signs. Syntactically they can function as pronominal core arguments of a clause (see Puupponen et al. 2015). When pointing with the whole head one may, for example, refer to the addressee in a conversation (Puupponen et al. 2015) or objects present in the communicative situation. In those occurrences identified in the data in Puupponen et al. (2015), they were nod-like movements which include tilting the head sagittally or sideways while pointing in the same direction as the gaze. In the study, referring to specific referents by pointing with the head was less frequent than other types of head movements (Puupponen et al. 2015). However, in different types of communicative situations signers may presumably point with their head in multiple different ways. It is suggested here, as a hypothesis for future research, that pointing with the head emerges more often as giving emphasis to, or as a consequence of, manual movements in indexical signs rather than independently without manual signs. Another potential context for independently occurring head pointing may be situations in which the hands of the signer are in other ways occupied. Independent pointing movements of the head are presumed to be more common in discourse data than in narratives, but in order to throw more light on this, pointing with the head should be investigated further with data consisting of multiple different discourse genres, including discussions on both present and imaginary referents and reference to locations both near and far from the interactants.

Indicating a referent can also be done with sideways movements of the head while connecting the content of manually produced signs to imaginary referents established earlier in the discourse. The signer may, for example, change the orientation of the face along with the eye gaze, tilt the head sideways, or lean the whole upper body sideways, or produce a combination of these movement features. In some of the movements, the orientation of the face changes according to the direction of the gaze. These are referred to as gaze-aligned movements (see also Enfield 2009). In these movements the head does not independently point as if projecting a vector to the object to which the signer is guiding attention. The direction of the gaze points while the movement of the head aligns with the gaze and potentially emphasizes the pointing. To this may be added a turning of the shoulders, torso or the whole body in the same direction as the gaze. When manual signs are produced during this action, their interpretation is anchored to the reference point of the pointing (introduced or pre-established). In gaze-aligned movements it is not easy to define whether the gaze and head movement direct attention to something or draw something to someone’s attention.

In some movements of the head or the whole upper body, the head moves towards an introduced or previously established location (i.e. an imaginary referent) while the face and gaze may be oriented towards an addressee (as in Figure 4) or to the imaginary referent. As with gaze-aligned movements, these movements anchor the simultaneously occurring manually signed contents to the reference point of an imaginary referent (introduced, re-introduced or maintained). In the head movement typology, this type of indication of a referent with the head is referred to as placing. With placing movement I refer to a movement which, instead of projecting a vector in the direction of a referent, “actually positions a temporal [or other] entity in space” (Cooperrider & Nunez 2009: 190; brackets added to the original quote by the writer). This definition is connected to Clark (2003), who distinguished between directing someone’s attention to something (i.e. pointing) and offering something for someone’s attention (i.e. placing).⁶

Figure 4

An example of an utterance in which the head tilts towards an imaginary referent (young deaf people), specified in the signing space earlier in the discourse, and connects the signed content to this theme (see also Puupponen 2012). The head turn during the tilt shows exclusion and it is analyzed here as enacting. SLM_2011_00:00:18.

Placing movements of the head, such as the one in Figure 4, are often longer in duration than pointing movements identifying referents. However, differences in the average durations of these movement types is an empirical question for future research. The anchoring of manually signed content may be done while the head moves or turns sideways, or there may be a sideways hold or a static position of the head during which the content is produced. They are not used for explicit pronominal reference and therefore they do not function as core arguments of a clause. They are, however, used together with manual pointing and pointing with the gaze to track reference in discourse (see also Lackner 2017). They may be optional, as indexical manual signs perform the same function, but they are presumed to occur quite often: the way the discourse is indexically organized is shown in the whole body of the signer, not only in the hands. On the other hand, they may also include enactment (as is the case in Figure 4). In general, it is suggested here that in many cases the difference between pointing and placing functions may not be clear cut. When it comes to movements of the head and eye gaze, in the same way as moving towards something may not be easy to distinguish from projecting a vector in the direction of something, directing attention to something or bringing something to someone’s attention may not always be distinguishable notions. The interplay between gaze and head movements as well as their indexical functions is a matter which should be dealt with in more detail in future research.

3.2.3 Indicating discourse structure

Head movements which do not indicate referents but only indicate discourse structure organize and parse texts and conversations by binding elements together into continuous stretches, by showing points of transition between these stretches, or by emphasizing elements during the stretches. In the head movement typology these are referred to as parsing movements, separative movements and emphatic movements, and they occur either together with movements of the torso or without them. Parsing movements bind elements together into stretches of discourse without placing or pointing (Figure 5). They are simply movement contours which bind together manual signs in relation to the syntactic structure and organization of the discourse (phrases, clauses, sentences, text episodes, turns in conversation). In sign language linguistics, these types of movements have been previously referred to as prosodic “domain markers” (e.g. Wilbur 2000; Pfau & Quer 2010; Puupponen et al. 2015; see also Sandler 2012). Formally they may not differ from placing movements: they are sagittal and sideways head movements which are relatively long in duration and may include a static hold or position of the head. In FinSL (Puupponen 2012; Jantunen et al. 2016a; Puupponen 2018) and Israeli sign language (Sandler 2011), for example, the parsing function has been associated especially with sideways directed movements of the head and upper body, although sagittal parsing is not uncommon. It should be emphasized, however, that head and body movements do not bind elements together on their own. Instead, parsing is done together with the kinematic features of manual movements as well as with the timing of different facial expressions (e.g. Ormel & Crasborn 2012; Sandler 2012).

Figure 5

An example of a sequence of utterances in a narrative in which a parsing movement of the head and upper body binds together manual signs and shows the area of a metacomment on the content of the story.⁷ CFINSL_008_05_00:05:43.

As mentioned in Section 3.1, movements of the head may also create contrast between elements in signed discourse (Figure 6). In this case the head (or the whole upper body) moves, for example, sideways while the content is being signed, after which the head comes back and moves again while content is signed that is in an oppositional, alternative, corrective or comparative relation with the previously signed content (see also Johnston 1992; Wilbur & Patschke 1999; van der Kooij & Crasborn 2006; Puupponen et al. 2015; Jantunen 2016). They are indexical embodied visualizations of a relation between two or more entities, and they may introduce referents which are referred to with anaphoric pointing later in the discourse. As in the example in Figure 6, the contrastive relation always includes at least two elements which are contrasted, that is, it always involves a relation of two entities or events. These contrasted elements may be expressed manually with only single signs or with whole sentences, the choice naturally affecting the duration of the different contrastive head movements. These manual elements may be contrasted with only head movements or also with manual conjunctions (see e.g. Jantunen 2016). Contrastive movements usually include two or three sagittal and/or sideways movement sequences (including the retraction phases of the movements), depending on the number of elements that are put in a contrastive relation. The movement sequences may be produced in distinctly different or even opposite directions, or in similar directions but with tilts to opposite sides. Further, when contrasting only two things, the head movement may emerge with only one of them, as is the case in the example in Figure 6. This is enough to create a perceptible distinction between the two things.

Figure 6

An example of an utterance in which movements of the head and upper body signal contrast between manually signed elements. The 3rd gloss from the right is FINLAND-SWEDISH and the abbreviation at the end of the 4^th gloss, ‘h2’ refers to the non-dominant hand. CFINSL_008_06_00:14:50.

Contrastive movements are parsing movements with a particular function. In this case their indexicality may identify referents (placing) as well as discourse structure (parsing), although they do not necessarily have to include a placing function. The temporal placement of the movement is enough to create a contrast. Furthermore, as was pointed out earlier, it can be argued that signaling opposition, alternatives and comparison by means of placing movements to opposite sides of the signing space invokes association of analogy (i.e. iconicity) as well as indexical signification. The stance taken in this paper is that this is one type of iconicity, and that contrastive movements are a good example of how different semiotic grounds come together in actions of the head.

Movements separating stretches of signing emerge on the boundaries between clauses, sentences, text episodes or turns in conversation. They are movements that are relatively short in duration – such as single head nods – and indicate linear transitions between signed sequences (Figure 7). For example, in a study comparing corpus narratives of Finnish and Swedish sign languages, a tendency was found for single head nods to align with syntactic boundaries and to occur sentence-finally in both languages (Puupponen et al. 2015; 2016). In sign language linguistics, these types of elements have traditionally been referred to as prosodic “boundary markers” or “edge markers” (e.g. Wilbur 2000; Pfau & Quer 2010; Puupponen et al. 2015). Stretches of discourse are also separated simply by the transitions between parsing head and torso movements, that is, they are not always pointed out by distinct movement events such as nods. The holds and changes of direction between longer movement contours are in themselves elements that organize the signing string into perceptible chunks, just like phonetic features which indicate word and sentence boundaries in speech.

Figure 7

An example of a separative head nod indicating a transition between two stretches of discourse. CFINSL_008_05_00:06:40.

Finally, in the head movement typology, many different types and forms of movements are classified as emphatic when they reinforce manual movements, and hence also the content conveyed by those manual signs. They may be produced along the movement path of manual signs, or in the opposite direction (hands and head moving towards or away from each other), which intensifies the manual movements. It is suggested here, as a hypothesis for further investigation, that emphatic head movements may be reflex-like consequences of manual motion or movements involving more control in their production and interpretation (see Enfield 2009; Kockelman 2005; Section 4.1) and underlining the content that is being signed manually (Figure 8).

Figure 8

An example of head movements emphasizing manually signed content. CFINSL_008_06_00:05:54.

In sum, parsing, separative, and emphatic movements all indicate the structure of larger episodes of discourse or smaller syntactic sequences. They guide one’s attention to the organization of the signed content and signal relations between textual elements. It is suggested here, as a hypothesis for future research, that parsing and separative movements may include less control in their production and interpretation, while emphatic movements may be more controlled, depending on the situation in which they occur. This issue will be discussed further in Section 4.1 (see Kockelman 2005; Enfield 2009). Some parsing movements and contrastive movements, presented earlier, may also be considered emphatic. This exemplifies the fact that the typology presented in this paper describes head movements through prototypes, not mutually exclusive categories. Different functions in the signification made with head movements may overlap.

3.3 Symbolic features in head movements

The symbolicity of head movements is most evident in negative headshakes and single nods or repeated nodding of the head with positive functions, which form more conventional form-function pairs than primarily iconic or indexical head movements. Headshakes and nods are globally widespread gestures frequent in the communication of both signers and speakers. Although there are observations of some variation in their functions across the world (e.g. LaBarre 1947; Samovar et al. 2007), they are somewhat conventionalized signs for negative (e.g. disapproval, disbelief, negation, grief) and affirmative meanings (approval, agreement, understanding, assuring). In the data from which the current paper draws, they may occur together with manual signs (Figure 9) or independently as responses to questions or as backchannelling cues. They have a more decontextualized meaning-like function than other types of head movements discussed in this paper, that is, their interpretation is not only context-specific and they can be considered as tokens of types.

Figure 9

An example of a headshake that changes the meaning of a sentence from affirmative to negative together with the facial expression. CFINSL_008_06_00:19:50.

Although the form-function connections in negative headshakes and affirmative nodding show more conventionality than in iconic or indexical head movements, this does not mean that their functions are clear cut. Headshakes, for instance, may convey different types of meaning related to negation or surprise: disbelief, hesitation, doubt, downplaying, disagreement, negation, sorrow, sympathy towards a negative issue, and so on (Kendon 2002; Johnston 2018; Puhto 2018). When a signer signals something negative with a headshake, this may be associated with the negation of a proposition in the discourse or it may convey meanings related to the way the signer’s personal feelings, thoughts or attitudes are positioned against what is said in the discourse (Puupponen 2012; Johnston 2018). Therefore, when we view symbolic head movements as semiotic signs, it is evident that the object to which they connect are not clear cut, but rather a gradient field of different types of functions that are more or less closely connected.

In addition, symbolic head movements are not only symbolic: the connection between the sign and its object is indexical for both of them. Negative shakes and positive nods are indices which show different degrees of conventionalization. For example, the rotation of the head in a headshake may be argued to have its origin in a gesture of rejection, a reflex that is already typical in infancy (e.g. a child avoiding a spoon while being fed). In the 19^th century Charles Darwin (1872) suggested that nodding and turning the head have their roots in the way babies lean towards milk (nodding) or reject it (turning). In addition to cyclical shaking movements, this type of function (i.e. a physical act of rejection) may form the basis for different types of exclusive and negative single head turns found in different sign languages (see Figure 4). On the other hand, it has been suggested that head turns stem from avoiding eye contact (Stern & Bender 1974). With relation to the indexical origin of head nods, they have been argued to have their roots in the act of bowing (Morris 1994) and in the development of reptiles already 280 million years ago (Givens 2013). Some scholars have also argued that head nods and head turns may be traced back to human mirror systems and their effect on babies’ abilities to imitate their parents’ movements and emotions from early on (Meltzoff 2002; Braten & Colwyn 2007; Thagard 2010). The current paper argues for the view that symbolic head movements in sign languages are schematized indexical signs which can be said to differ in their degree of conventionality and to vary in their meaning according to the communicative situation.

Finally, symbolicity occurs also in other head movement types, although not in the Peircean sense of legisigns. As with speakers’ manual gestures, so too with signers’ nonmanual actions the conventionality may lie in the “types of communicative action” (Enfield 2009; see also Kendon 1988; Okrent 2002), rather than requiring specific forms for specific functions. The ways in which iconic and indexical movements are produced have recognizable features through association and inductive processes even though they do not emerge as types for tokens in the same way as do shakes and nodding of the head. Their recognizability may involve understanding the habits of a single signer (i.e. entrenchment) or conventions shared among a group of signers. Nods and shakes, on the other hand, are easy to associate with convention because they convey meaning that can be understood roughly along the same line of interpretation, whatever the context.

3.4 Proportions of different grounds in head movements

As has been presented in the preceding sections, different types of head movements involve iconicity, indexicality and symbolicity in overlapping ways. For example, enacting and time-metaphor head movements include both iconic and indexical features, movements indicating referents by placing them in contrastive relations may have iconic features, and symbolicity emerges in different ways in different types of head movements. It may, however, be that these different strategies of signification emerge in different proportions in different head movement types. In order to demonstrate this, I will borrow a visualization suggested by Capirci (2018) for the iconic-indexical-symbolic relations of linguistic signs (Figure 10).

Figure 10

The proportions of different grounds in different head movement types (visualization according to Capirci 2018).

As suggested by Figure 10, the proportions of iconic (yellow), indexical (green), and symbolic (red) grounds are different in different types of head movements. While negative headshakes include a lot of indexicality and symbolicity, their iconicity lies mainly in the associational recognition of tokens to types (see Peirce 1903a; Silverstein 1955; Parmentier 1994; Kockelman 2005). With movements indicating discourse structure (parsing, separating, emphatic) and indicating referents (pointing, placing), the indexical ground is strong whereas iconicity lies in the association of the types of communicative action (together with other sign mediums) to the token sign events, that is, in recognizing their symbolic ground (see Enfield 2009). In addition, as pointed out earlier, in contrastive movements, for example, the spatial relations between the contrasted element may show also other sources of iconic association. As for enacting and time-metaphor movements, the iconic ground is strong but they include a lot of indexicality as well. The symbolicity of these movements lies in the recognition of patterns in the communicative action, that is, in connecting the token enactments and token time-metaphor movements to the “general manner of execution” (Enfield 2009: 19) in which they are done among signers. Finally, one very visible feature in Figure 10 is how redness—symbolicity—is not the most evident feature in all head movements. Symbolicity may not be the prime motor of this type of signification, but still head movements are significant. This, as I argue in Section 5, speaks for the need for a theory of language that embraces less symbolic and unconventional elements instead of pushing them to the periphery.

4.1 Semiotic complexity & pragmatic unity in signers’ actions

As presented in Section 2.5, in a semiotic analysis of signers’ head movements it should be noted that head movements in a given communicative situation are one part of a semiotically complex whole—a composite utterance (Enfield 2009)—consisting of several signs from different sign mediums and driven by a unified intentional social action (see ibid.). This applies for all head movements, also movements such as headshakes that are produced without any manual action, for example, in response to an interlocutor’s utterance (i.e. they may emerge together with eye contact, facial expressions, and so on). No signals produced with the head in natural discourse emerge in isolation and they cannot be interpreted as isolated entities.

There are, however, differences in what kind of components signals of the head are in clusters of semiotic features. Some of the head movements discussed in Section 3, such as a headshake can be interpreted as conveying meanings. Head movements indicating discourse structure, on the other hand, cannot in themselves be associated with a specific meaning. The function of a parsing or separating head movement comes about in its relation to the whole signed utterance, in its temporal alignment with the actions of the hands. This means that its functions emerge through a physical, actual relation—a co-occurrence or proximity of timing—which makes it an indexical sign (Peirce 1903a; Parmentier 1994; 2006; Kockelman 2005). A sentence-final head nod or a parsing movement of the head (or the whole upper body) organize the discourse by indicating changes or continuities in the structure along the lines of, ‘here is a change in the structure of discourse’ or ‘this is a continuous stretch of discourse’.

Semiotically, a separating head nod or a parsing sideways movement of the head are signs that do not have a clear and easily determined object (cf. Kockelman 2005). Instead, the object is an abstract notion that emerges in the co-occurrence of head movements with manual actions: ‘something is happening here that is distinguishable from what has come before and/or is subsequently going to come in the discourse’. This is in line with Kockelman’s (2005: 242) notion of the abstract object which is “less precise”, “less consistent” and does not necessitate “intentional states such as knowing” (see also Section 2.1). The interpretant of this type of a sign is the feeling or state that comes about when observing the sign, and which may be vaguer and more subconscious than consciously acknowledged: ‘the processing and interpretation of these contents is smoother and less obstructed because this signal organizes the structure of discourse’. As a hypothesis for further investigation, it is suggested here that a head movement indicating discourse structure involves less control in both its interpretation and its production, and that it therefore shows a lower degree of agency than, for example, a headshake (Kockelman 2005; Enfield 2009). It may be that the existence of such signs becomes evident only when there are perceivable errors or disfluencies in the signing as a result of the atypical use of these signals.

These types of signals are similar to what Enfield (2009) calls implicit indexical resolution. Although Enfield seems to use the term to refer to indexical signs with an implicit placing function (see also Clark 2003)—that is, to signs that link other signs in an utterance to specific points in space with mere placement (e.g. a no-smoking sign)—it is partly applicable to signs indicating discourse structure without a placing function. Both types of signs (i.e. a no-smoking sign and, e.g. a separative head nod) are implicit in their indexicality, that is, they come about through the physical situation in which they emerge or to which they are positioned without any explicit indexical content. Despite their implicitness, both sign types are, however, indexically connected to other signs in the semiotic cluster. A no-smoking sign links the interpretation of the content of the sign to a specific location without actually pointing to it (e.g. with an arrow). A head movement indicating discourse structure is linked to the linear structure of signed utterances, or larger stretches of discourse, by mere temporal placement, the proximity of timing. The interpretation of these head movements is interrelated with the holistic interpretation of the whole composite utterance and, further, with the textual entirety that they build together. This unity in interpretation is possible because of contextual association (Enfield 2009), among other factors. The timing and other indexical proximity of these signals are triggers that guide the interpreter to assume that they are “part of one signifying action” (Enfield 2009: 16) together with other signs in the utterance. To summarize, as a hypothesis it is suggested here that head movements include different degrees of control, as has been found to be the case with, for example, eye-blinks in signed discourse (e.g. Wilbur 2000). Movements indicating discourse structure involve less control than, for example, more conventionalized head movements such as headshakes or head movements indicating referents. It is also suggested that different head movement types differ in how dependent their interpretation is on other co-occurring signs in other mediums. While a headshake is connected to its composite utterance, its function does not rely on other signs to the same extent as the function of a separative head nod. Nonetheless, they are all part of intentional communicative actions which, according to Enfield (2005: 17), are driven by a pragmatic unity: the “unified, single, addressed utterance–meaning”.

4.2 Signification with the head and other parts of a signer’s body: Central semiotic features

It is not surprising that different sign mediums in sign language discourse form a whole, and that different body parts share features in terms of what types of (and how) signs are created with them. However, I argue that there are differences in the central semiotic features signaled with different body parts. This semiotic versatility (Wagner et al. 2014: 209) is a feature that is evident in the communication and interaction of speakers as well as signers. Hands convey information regarding shape and size better than speech, while the face is well suited for expressing feelings and attitudes (Wagner et al. 2014). I argue, further, that in addition to approaching the signification of signers as a holistic social action that embodies a cluster of semiotic signs, it is important to discuss the potential differences in the pivotal features of these sign mediums (hands, face, head, body). This requires both that we investigate signification while focusing on signers’ specific body part(s) and that we investigate the interrelations of these different parts of signing. In this way we might find out whether or not the central features of different sign mediums in signer’s communicative actions are similar and learn more about the roles of and interplay between different nonmanual signals in signed discourse.

The view taken in this paper on the semiotic versatility of signers’ actions is summarized in Table 1 with regard to each sign medium. As is shown in the table, manual signs include conventional signs, unconventional signs and hybrids of the two (see Enfield 2009). These signs show different levels of iconicity, indexicality and symbolicity (as defined in the current paper), and it is not uncommon at all for signs to present all these features simultaneously (Ferrara & Hodge 2018).⁹ Manual signs have iconic features which require the analogical association of the similarities between qualities of the sign and qualities of its object. A handshape consisting of a stretched index finger used for referring to a small object such as a pen is an example of an iconic feature in a manual sign, as is the signaling of size with tracing movements or qualities of actions with manual kinematics. On the other hand, pointing and placing movements (indicating referents) and some manual kinematics and manual signs (indicating discourse structure) are examples of the indexicality of manual signs. The symbolicity of manual actions can be seen in the shared norms among signers of a language that have emerged through frequency of use and which enable lexicalization processes. Both description with partly lexical signs and enactment are common strategies through which fully lexical signs may emerge in a sign language (e.g. signs that describe whole entities, shape, size or handling) (e.g. Johnston & Schembri 1999; 2010; Liddell 2003; Johnston & Ferrara 2012; Ferrara & Halvorsen 2018; Ferrara & Hodge 2018). Especially interesting is the fact that both of these iconic strategies have a two-way connection to the established lexicon: while signs may conventionalize and become fully lexical signs, this process may move in the opposite direction. That is, fully lexical signs may “turn back” to a more gradient type of signing, such as enactment, a process which is often referred to as delexicalization (Johnston & Schembri 1999; Cormier et al. 2012; Johnston & Ferrara 2012). Johnston & Ferrara (2012: 237) refer to this phenomenon as the “two faces of the sign”. To conclude, the manual elements of a sign language form gradient categories (see e.g. Jantunen 2017). This is the result of the different strategies available in discourse.

A signer’s face conveys a lot of information as well. All in all, signers’ facial expressions are elements that characterize something. They convey information about the qualities of referents and their actions, emotions or attitudes. For this reason, parallels have been drawn between facial expressions and grammatical classes of spoken languages, such as adverbs and adjectives (mm. Wilbur 2000; Pfau & Quer 2010; Herrmann & Pendzich 2014). These labels have been given in particular to mouth actions—elements produced with the lower parts of the face—and to whole facial expressions in general. However, a recent study (Johnston et al. 2015) has challenged the view that elements produced with the face are “linguistic markers with a grammatical function” (Herrmann & Pendzich 2014: 2149). When the form-function pairing of signers’ mouth actions was studied from a large set of video material, they were found to show patterning, but not to function as conventionalized grammatical markers (Johnston et al. 2015). Instead, their use was found to be idiosyncratic, and different types of mouth actions were placed on a continuum from indexical to iconic, unconventional and finally potentially grammatical (Johnston et al. 2015). Upper parts of the face, on the other hand, have a range of indexical functions. Firstly, the direction of the eye gaze is used for pointing at present or imaginary referents or placing imaginary referents to the attention of interlocutors (e.g. Wilbur 2000; Schembri et al. 2018). Secondly, eye-brow movements and eye aperture may parse and emphasize elements, such as old and new information, or the lack of information (see e.g. Wilbur 2000; Sandler 2012). Finally, some eye-blinks, biologically driven actions for keeping the eyes moist, separate stretches of signing by indicating transitions between them (Wilbur 2000; Sandler 2012).

All in all, elements produced with a signer’s face form a semiotically interesting research topic. Although different parts of the face may be distinguished on the basis of the elements they produce, the facial expression of a signer is often a unified whole which is interpreted imagistically in its entirety. Apart from the eye gaze and blinking, the upper and lower parts of the face should in this case be investigated as a whole, not only locally. Eye gaze and blinking, on the other hand, may function more independently also during more unified facial expressions (pointing and separating functions). It may be that the signals produced with the face vary in how dependent their interpretation is on other co-occurring signs in other mediums, in the same way as was suggested for different head movements. Finally, it should be noted that when it comes to actions of the face, it is difficult to distinguish between enacting, the non-enacting description of qualities of a referent, and the expression of the signer’s own feelings and attitudes (see e.g. de Vos et al. 2009). The face forms a semiotically complex sign medium, which should in some cases be approached as a cluster of smaller and more specific semiotic mediums.

As can be seen from Table 1, the semiotic repertoire of head movements is in some ways more limited than that of the hands and face. The capacity to reflect or imitate visible features in the world is not as diverse for signs produced with the head as it is for signs produced with the hands and the face. The physiology and anatomy of the head (see Puupponen 2018) alone limit the potential for imitation in head movements, and actions of the head are often more approximate than those of the hands and face. Head movements may depict referents, their actions and their qualities, but this is mostly enactment. Non-enacting iconic features are not as common. Furthermore, symbolicity (as defined in the current paper) is not as prominent in the actions of the head as in that of the hands, as was presented in Section 3. Head movements seem to rarely form types for tokens, they involve more analogical association and spatio-temporal contiguity, and they include different degrees of control in their production and interpretation. The conventionality of head movements emerges most probably in the recognizable manner in which certain communicative actions are produced.

Compared with head movements, the actions of the (upper) body have an even more restricted variation in semiotic functions. As is the case with signs produced with the head, also body movements parse and emphasize elements, indicate referents by placing, and enact referents. Symbolic types for tokens are presumably rare and may emerge only in movements such as shrugging the shoulders. However, although the semiotic repertoire in signs’ representative nature is more restricted than that of the hands and face, this does not mean that the body does not convey a lot of information. The diversity and richness of actions of the body is shown, for example, in the ways in which signers and speakers convey information related to their identity, state of mind, emotions and social distance with body movements that may not be intended as communicative actions (see e.g. Streeck et al. 2011).

The semiotic versatility of actions of different parts of signers’ bodies may be summarized as follows: (i) Hands produce more fully lexical symbolic signs (i.e. types for tokens) than other parts of the body, indicate referents and discourse structure, signal one’s own emotions and attitudes, and show semiotic flexibility (signs may emerge both as lexical signs and as more gradient and unconventional enacting); (ii) The face can enact, describe without enacting, indicate referents, and indicate discourse structure, and some signals may become more conventional symbolic signs; (iii) The head can indicate referents, indicate discourse structure, enact referents, and connect to (time) metaphors but symbolic types for tokens are rare, as is non-enacting description, (iv) The (upper) body indicates referents and discourse structure with a slightly smaller repertoire than the head, enacts referents, and connects to time metaphors. As can be seen from the summary, indexicality and iconicity are everywhere while symbolic types for tokens are primarily produced with the hands. As a result, I suggest that the potential for communicative actions of different parts of the body to produce symbolic signs in the sense of Peircean legisigns is strongest for the hands, possible for the face, rare for the head and unusual for the upper body. Thus, the signification with different parts of signers’ bodies forms a continuum, from hands to body, between an abundance of symbolic content and very little symbolic content.

These differences in the central semiotic features of actions of different parts of the signer’s body come back to the physical characteristics of these different sign mediums. There are differences in what kind of information can easily be conveyed with different parts of the body. Borrowing from Enfield (2009: 18), hand movements “are well suited to iconic–indexical meaning thanks to their rich potential for sharing perceptible qualities in common with physical objects and events”. As Wagner et al. (2014) pointed out, the face is a convenient medium for signaling emotions and attitudes. However, tracing the movements or shape of entities is not a feature which is enabled by the physical characteristics of the face; the quantity or volume of entities, on the other hand, may be signaled with, for example, different mouth actions. The physical characteristics of the head and body are not well suited for signals that share physical qualities with physical objects and events, except for showing the actions and existence of referents through enactment. However, as with the hands and eye gaze, with the head and the body one may produce signals which point or place, for example. In this function one may also see differences between the different mediums. As the size of the body part that produces the pointing action becomes larger, so too the object of the pointing becomes less precise. Pointing with the head or the whole body is in many cases—especially when referring to referents that are not physically present—more approximate than pointing with the index finger or the whole hand/arm. Furthermore, while placing movements of the hands function well for indicating precise spatial relations between entities (micro-level), placing movements of the head or the whole body are better suited for large scale thematic, textual and syntactic sequencing (macro-level). Finally, semiotic versatility in the signs produced with different body parts applies to speakers as well as signers. When looking at, for example, the comprehensive literature review on co-speech head gesturing carried out by Wagner et al. (2014), there are evident similarities between speakers’ and signers’ head movements, and comparing their forms and functions raises many interesting questions for future research.