1.1 Prior research on emoji

In recent years, emoji have been investigated from a wide variety of perspectives: In addition to an emerging body of linguistic research, emoji have been researched in fields as varied as computer science (see e.g. LeCompte & Chen, 2017 on sentiment analysis), marketing research and communication (see e.g. Luangrath et al. 2017, see also Jaeger et al. 2019 on emoji valence), psychology (see e.g. Li et al. 2018 on personality traits and emoji use), health communication (see e.g. Troiano & Nante 2018), and education (see e.g. Dunlap et al. 2016 on emoji in online learning). We refer readers to Bai et al. (2019) for a recent overview.

Most relevantly for us, emoji have also attracted the attention of a growing number of linguists, as already noted above (see also Evans 2017 for recent discussion). From a linguistic perspective, researchers have investigated the combinatorial properties of emoji sequences, e.g. whether strings of emoji have grammatical/syntactic properties. For example, in important foundational work incorporating both face emoji and activity emoji, Cohn et al. (2019) provide experimental evidence indicating that emoji have only restricted combinatorial properties and do not have grammatical structure of their own. (However, see also Gerke & Storoshenko 2018 for evidence that people’s native language may influence their emoji ordering preferences.) More closely related to our work is research on the relation between emoji and the linguistic elements that they typically occur with. One way of approaching this relation is represented by the work of Gawne & McCulloch (2019), who analyze some emoji as digital counterparts of the gestures and facial expressions that accompany spoken language. Another approach relevant for our aim of investigating the relation between emoji and text is Maier’s (2021) proposal, which analyzes face emoji and facial expressions as use-conditional items (see Gutzmann 2013) and which we review in Section 4.2. Our research builds on these prior studies, and systematically explores the behavior of both face and activity emoji in contexts involving the interplay of linguistic strings (words) and emoji.

1.2 Aims of this work

Our research positions itself in the tradition of emerging formal semantic research on digital communication (e.g. Bücking & Rau 2013). In the present paper, we propose that a semantic distinction needs to be made between [i.] face emoji, which typically convey affective information, and [ii.] activity emoji, which convey information about actions and activities. More specifically, we show that face emoji and activity emoji differ in their linguistic properties, with face emoji incorporating perspective dependence, with a preference for first-person indexicality, while activity emoji incorporate anaphoricity in connection with the inferred agent of the emoji-related activity (or holder of the emoji-related property). In the rest of this paper, we take initial steps towards a formal semantic analysis of these two subsets of emoji.

In terms of empirical data, our strategy is as follows. In this paper, we focus on using both constructed and naturally occurring examples to establish solid intuitions, which are captured by initial hypotheses. In current and on-going work, we test these hypotheses experimentally (using psycholinguistic methods such as the forced-choice task in Kaiser & Grosz 2021), thereby corroborating the validity of using constructed examples and introspective intuitions for emoji. It is worth pointing out at this stage that many of the examples used in this paper are naturally-occurring examples from Twitter (marked with a ^[twitter] superscript). When citing text from social media, one is faced with questions of ethics and privacy (see e.g. Ayers et al. 2018, Tatman 2018). In this paper, we remove authors’ user names and suppress URLs, but keep the posts as they are, to ensure the linguistic integrity of the examples; all personal names within Twitter posts were replaced with John, Smith or John Smith, and @-signs were removed. (None of our examples concern sensitive topics or private information.) This cautious approach to protecting personal data entails that Twitter examples only show up in a reverse search if authors’ account settings permit, and disappear if authors delete posts, or make their account private. It is not unusual for web corpus data to disappear over time; many URLs in the British WaCky Web Corpus ukWaC (Baroni et al. 2009) are broken links as of December 2022.⁵

If the original Twitter post contained multiple emoji, we maintain them in our examples for reasons of completeness (unless marked otherwise). However, the present work focuses on message-final emoji and on the first emoji in a sequence. We leave the intriguing question of emoji sequence interpretation for future work. We chose to focus on message-final emoji because several empirical studies show that they are the most frequent (e.g. Garrison et al. 2011; Novak et al. 2015; Al Rashdi 2015; Cramer et al. 2016; Sampietro 2016; Na’aman et al. 2017; Seyednezhad et al. 2018). Emoji can also occur in other positions (e.g. message-medially and even message-initially), but these positions are less frequent; we leave them for future work as well.

To establish individual emoji meanings in a non-subjective way, we draw on: (i) the form of the emoji, i.e. what they resemble, (ii) the non-academic reference website Emojipedia,⁶ and (iii) emoji norming studies, emerging in increasing numbers (e.g. Ferré et al. 2022). For emoji-text combinations, we use introspective intuitions of emoji users, which are later verified in controlled experiments (e.g. intuitions from Grosz et al. 2023 were corroborated experimentally in Bjertnes 2022).

Our empirical claims, in a nutshell, are as follows. As discussed in Section 2, we propose that activity emoji (e.g. , , ) incorporate anaphoricity, in the descriptive sense of referring back to a previously-mentioned referent. To illustrate what we mean by anaphoricity, consider examples (1a-b), which are identical except for the verb (impressed vs. admired). In (1a), the basketball emoji is likely to convey that Sue impressed Ann because of how she (Sue) played basketball. In contrast, in (1b), the emoji will probably be interpreted as conveying that Sue admired Ann because of how Ann played basketball. That is, connected to the fact that the basketball emoji provides an explanation for the preceding predicate (being impressed or admiring), the inferred agent of basketball-playing is different in (1a) and (1b). As we show in Section 2.2, we can straightforwardly derive this difference from prior work on pronoun resolution if we assume that activity emoji [i.] involve anaphoricity and [ii.] are sensitive to discourse relations (e.g. Explanation, Elaboration, Resemblance) in ways that parallel the sensitivity observed in prior work on pronoun resolution. (We view our observations as being in line with the claims of Hobbs 1979 and Kehler 2002 that pronoun resolution in the linguistic domain is a side-effect of general inferencing processes about coherence.) Informally speaking, the basketball emoji in (1a) is anaphorically connected to Sue, whereas the basketball emoji in (1b) is anaphorically connected to Ann.

We propose that activity emoji denote separate discourse units, which freely associate to the text via suitable discourse relations, as exemplified in (2) and (3) for Explanation and Elaboration relations. (Throughout this paper, we use the wave arrow ‘⤳’ to mark meaning inferences without committing to a particular status of the inference, e.g. whether it is entailed, presupposed or implicated.) We start with an informal understanding of discourse relations: for example, Elaboration describes a situation where the events in a discourse unit β are sub-events of the events in a (preceding) discourse unit α (i.e. β elaborates on α); see Asher & Lascarides (2003: 159).

In contrast to activity emoji, we propose that face emoji (e.g. , , ) (e.g. Riordan 2017, Jaeger et al. 2019) — and presumably also other affective emoji, such as , , — exhibit perspective dependence. To see this, let’s consider (4a-b).

Examples (4a-b) are the same as (1a-b), except that the basketball emoji has been replaced by the ‘surprised face’ emoji. Now, we no longer see the referential switch from Sue to Ann that we observed in (1a-b). Instead, in both cases there is a strong bias to interpret the surprised face emoji as reflecting the emotional state of the first-person author of the entire message, rather than Sue or Ann. This default bias for first-person indexicality is known to be a property of expressives (e.g. damn, see Potts 2007; Lasersohn 2007; Amaral et al 2007; Harris & Potts 2009). As we show in Section 3, we propose that face emoji have lexical entries along the lines of what is exemplified in (5) for the ‘surprised face’ emoji (based on the approach to expressive presuppositions from Sauerland 2007 and Schlenker 2007), where w₀ is the utterance situation, and author₀ the author in the utterance situation; both have their values assigned by a contextual assignment function g to allow for reported utterance situations. Our lexical entries do not assume any syntax or direct compositionality for emoji; the notation in (5) is based on Egg’s (2012) notation for discourse particles, where p arguments are retrieved from the context.^7,8 As a consequence of this indirect (anaphoric) connection between face emoji and their propositional argument p, it is worth highlighting that face emoji are strictly speaking also anaphoric elements. However, since we focus on the different ways in which face emoji and activity emoji determine their attitude holder and agent, respectively, we reserve the term anaphoric(ity) for discussions of how activity emoji retrieve the agent of the denoted event — which we argue to be similar to the way in which pronouns retrieve their referents.

It is worth noting that face emoji are among the most frequently used emoji; as a group, they are used more often than activity emoji (see e.g. Emojitracker 2021). This is presumably related to their resemblance to facial expressions, which play a central role in human communication. There exists a large literature on facial expressions (e.g. Tomkins & McCarter 1964; Ekman & Friesen & Ellsworth 1972; Russell & Fernández-Dols 1997; Fernández-Dols & Russell 2017 i.a.), but an in-depth discussion of the nature of the relation between face emoji and facial expressions (see, e.g., Fugate & Franco 2021, but also Maier 2021) is beyond the scope of this paper.

The structure of this paper is as follows. Section 2 focuses on activity emoji and presents our analysis that emoji in this subclass involve anaphoricity and are related to the linguistic text that they accompany by means of discourse-level coherence relations. We also look at the scopal behavior of activity emoji, showing that while they are typically interpreted outside of the scope of the negation, the discourse relation at play makes a crucial contribution. In Section 3, we turn to face emoji. This section presents our proposal that the semantics of face emoji is similar to that of expressives (e.g. damn, friggin’, yay, boo) and that this subclass of emoji exhibits perspective sensitivity: it is by default anchored to the first-person sender (akin to the first-person orientation exhibited by expressives and other perspective-sensitive expressions). As regards scopal behavior, we show that face emoji typically outscope negation, but suggest that ‘beat-related face emoji’ (a particular way in which face emoji are repeated throughout a part of the message) may have an emerging potential to be interpreted under the scope of negation. Section 4 brings together our findings for activity and face emoji. In Section 4, we also identify a question for future work that is beyond the scope of the present paper, namely how to determine the core meanings (i.e. the lexical entries) of emoji, and explore two possible approaches, the iconic (picture-based) approach and the lexicalist (convention-based) approach.

2.1 Activity emoji and discourse relations

Turning towards a formal implementation, our core proposal amounts to treating activity emoji as non-restrictive modifiers that denote an eventuality and/or property (Davidson 1967). In thinking about the relation between an activity emoji and the text that it accompanies, we draw inspiration from prior work on modifiers such as the gerunds playing the violin and being an artist in (6b) and (7b) (such gerunds being a paradigm case of free adjuncts, see Stump 1985: 42, though we do not claim that activity emoji should be analyzed in precisely the same way as gerunds). These gerunds act as independent utterances (see Zobel 2019, i.a.) that are connected to their ‘host clause’ by virtue of salient discourse relations (see Hobbs 1979; Lascarides & Asher 1993; Kehler 2002; Asher & Lascarides 2003). In other words, there is no direct compositionality between activity emoji and the accompanying text. It is not central to the claims in this paper whether or not activity emoji should receive a semantic analysis that treats them exactly like free adjuncts. However, the inspiration that we draw on is brought out by using gerunds in the paraphrases.

Our paraphrases intentionally highlight ambiguities such as playing the violin (activity) vs. being a violinist (state). In fact, as pointed out to us by Emar Maier and Louise McNally, the actual semantics of / may be more minimal, e.g. ‘(there is a) violin/artist’; crucially, emoji-containing examples would routinely require such a minimal semantics to be coerced into something eventive or stative, to yield (6b) or (7b). We view this kind of simple semantics as advantageous, as it seems that meanings associated with particular emoji can vary considerably but are nevertheless constrained by context and real-world knowledge; emoji-text interaction could well be as broad yet contextually restricted as noun-noun compounding. We leave the details for future research (see also discussion in Section 4), but note that similar kinds of ambiguities are well-attested in natural language expressions, see e.g. Asher (2011).

We propose that an approach based on discourse relations can be applied systematically to the relation between activity emoji and the texts they accompany. This is exemplified in (8)-(12) for a (non-exhaustive) set of five different discourse relations (Explanation, Elaboration, Narration, Background, and Result). For ease of exposition, our presentation of each of these examples is structured in the same way: (8a) names and defines the discourse relation, (8b) presents a natural-language example (all from Lascarides & Asher 1993: 55), (8c) contains a naturalistic emoji example from Twitter, and (8d) provides a paraphrase of (8c) in line with (8a). When discourse relations connect entities such as clauses or emoji, these entities can be referred to with the term discourse unit. The Greek letter α refers to the first unit (which is Max fell in (8b) and I really admire this man in (8c)) and the letter β refers to the second unit (which is John pushed him in (8b) and the activity emoji in (8c)). Examples (9)-(12) are parallel.

Note that, in prior work on speech-accompanying gestures, Lascarides & Stone (2009) and Hunter (2019) argue that such gestures are integrated with the accompanying text on the basis of suitable discourse relations. This idea parallels our proposal for emoji, and thus provides additional support for our approach. Maier (2021), who takes a different approach to the lexical entries of emoji from the one we propose, shares our view that discourse relations are operational in establishing the meanings of certain text-emoji combinations. It is also worth emphasizing that our approach allows for the emoji-text connection to be unconstrained and potentially ambiguous to the receiver of the message. For instance, consider (10c) and (11c), which we take to be illustrative of Narration and Result, respectively. In both examples, other discourse relations appear to be available: (10c) could be interpreted as Background, giving rise to the inference that SpareZ is where playing bowling and billiard happens; (11c) could illustrate Elaboration, i.e., rest day includes biking-related activities (e.g., if the sender of the message is competing in a cycling race). Note that this underspecification of emoji-text combinations mirrors that of text sequences: ‘John fell. He saw Mary.’ could be interpreted as Explanation, Narration, Background, etc. The flexibility in the emoji-text connection is exemplified in (13a). This example seems to be compatible with all five of the above discourse relations, as illustrated in (13b-f). This example also shows variability in the text (α) that the emoji comments on, which is ambiguous between the entire clause (so glad I stayed home today), (13b), and the embedded clause (I stayed home today), (13c-f).

2.2 Enter anaphoricity in activity emoji

We propose that activity emoji (such as and ) share a further property with free adjuncts such as gerunds (in addition to using discourse relations to connect to the accompanying text). They have an anaphoric component, relating to a previously-mentioned referent (as we saw in the Sue/Ann-basketball example (1)) or, in some cases, relating to an implicit agent (as we will see in (18b)). To illustrate this claim, consider (14a) (repeated from (3)), under the plausible reading where it involves an Elaboration discourse relation, as well as the Explanation example (15a) (repeated from (2)). A first attempt at a formalization of the integration of the emoji in (14a)/(15a) is given in (14b)/(15b) (loosely inspired by non-emoji work by Fabricius-Hansen & Haug 2012: 165 on gerunds). The symbol ≤ stands for ‘sub-event of’, i.e. e_i ≤ e_k symbolizes that e_i is part of a complex event e_k. The event e₁ in (14)/(15) is the event described by the text; e₂ is described by the emoji. Based on the discourse relation that connects them, a more complex event e results, with e₁ and e₂ as sub-events. In (14b), the agents x and y of the getting-ready event and the weight-lifting event are free variables that need to be contextually resolved (here, x and y are both identified with the author of the post); in parallel, x, y and z are free variables in (15b). While (14b-c)/(15b-c) are formal renderings of the emoji-text combinations in their entirety, we revisit the lexical entries of the emoji themselves in Section 4; to foreshadow that discussion, we remain agnostic with regards to the question of how much of (14b-c)/(15b-c) is due to the lexical meaning of the emoji, and how much is due to pragmatic enrichment.

Evidence for an anaphoric component in activity emoji (as modeled by the free variables in (14) and (15)) stems from effects such as implicit causality (e.g. Garvey & Caramazza 1974; Hartshorne & Snedeker 2013; Bott & Solstad 2014, and many others). The phenomenon of implicit causality is illustrated in (16) (coreference marked by bold type), and amounts to the following observation: an ambiguous pronoun in a causal adjunct (she in because she was hix¹¹) is preferably resolved towards the stimulus argument of a main-clause psych predicate. Since impressed is a stimulus-experiencer verb, she is preferably resolved towards Sue in (16a); by contrast, with the experiencer-stimulus verb admired in (16b), she is resolved towards Ann.

Crucially, we find exactly the same patterns of resolution with activity emoji when the intended discourse relation is Explanation. This is schematically illustrated in the constructed minimal pair in (17ab).

Naturalistic examples are in (18), with stimulus-experiencer verbs, and in (19), with experiencer-stimulus verbs. The examples in (18) echo the stimulus-experiencer pattern that we see in (16a) and (17a): The agent of the emoji event is identified with the stimulus of the preceding verb (e.g. the subject of impressed in (18a)). To unpack these examples, each of them contains a stimulus-experiencer verb (impress, disappoint, inspire, amaze, shock, surprise). In some cases, both arguments are overt, as in (18a), where the stimulus is Arsenal, and the experiencer is me, i.e. the author of the message. In other cases, both arguments are missing, as in (18b), which is a passivized truncated clause; here, the stimulus of disappoint is the implicit ‘agent’ of disappoint (‘disappointed by x’) and the experiencer is the dropped subject (‘I am [ready to be disappointed]’). In each of (18a-f), the agent of the emoji event is resolved towards the stimulus argument (overt or not) of the predicate in the text. A possible paraphrase for (18b) could thus be given as follows: ‘I am ready to be disappointed by someone, namely by the football-player(s)’. Our analysis of (18a) was spelled out in (15). See also Bott & Solstad (2014) on the connection between stimulus arguments (= individuals) and their activities, which are the cause of an experience in sentences with psych predicates.

Similarly, the examples in (19) corroborate the experiencer-stimulus pattern that we see in (16b) and (17b). Once again, in all of the examples in (19), the agent of the emoji event is identified with the stimulus of the preceding verb (e.g. the object of envy in (19a)). In (19a)), the stimulus of envy is identified with the referent of a lot of people, while the experiencer is identified with I, i.e. with the author; again, a paraphrase could be given as follows: ‘I envy a lot of people right now, namely those who play football’. As in (18), there are cases of truncation, such as (19b), which behave the same way.

In sum, we have argued that activity emoji serve as free-standing event descriptions, whose core argument is anaphoric, and which connect to the accompanying text through suitable discourse relations. This idea receives further support from experimental work by Kaiser & Grosz (2021) on the interpretation of activity emoji. As we will see in Section 3, face emoji exhibit a different behavior. However, let us first take a look at the scopal behavior of activity emoji, which provides further evidence for our claim that discourse relations play a key role with activity emoji.

2.3 Activity emoji and scope

So far, we have argued for a formal semantics of activity emoji, which can form the foundation for more intricate questions. Specifically, Pierini (2021) and Pasternak & Tieu (2022) investigate whether activity emoji can scopally interact with material inside the accompanying text. In the rest of this section, we proceed to revisit this question based on our new findings. In particular, we show that (i) activity emoji have a strong preference to scope over (or rather outside) all accompanying text, but (ii) if activity emoji combine with an Elaboration discourse relation, the appearance of scope interactions can be attested. Thus, these observations provide further evidence in favor of our claim that discourse relations play a key role in the interpretation of activity emoji. We will discuss the relation between face emoji and scope in Section 3.3.

3.1 Face emoji as expressive modifiers: basic proposal

Having argued that activity emoji have a semantic component similar to anaphoric elements and that their connection to accompanying text crucially depends on discourse relations, we now proceed to our proposal for face emoji. We argue that face emoji (and presumably affective emoji more generally) can be modeled on a par with expressives such as damn or friggin’ (e.g. Potts 2007), or even maybe more accurately, with emotive interjections such as yay, alas, ugh or boo (see, e.g. Goddard 2013; Gutzmann 2013; Rett 2021; Grosz et al. 2023). The core intuition is exemplified by (30). We argue that the emoji — called the “enraged face” emoji on Emojipedia (originally called the “pouting face” emoji) — in (30a) makes a contribution that is very similar to the expressive element friggin’ in (30b), or ugh in (30c). (Note that we are not claiming that (30a) and (30b-c) are identical in their meaning, simply that — as will become clear below — both the emoji and the expressives convey the author’s/speaker’s anger.)

Crucially, whether face emoji behave like expressives or interjections, we predict that they exhibit perspective dependence, which manifests in a preference for first-person indexicality. The default attitude holder of the subjective affect conveyed by a face emoji is the author of the message (see Potts 2007; Lasersohn 2007 for expressives in natural language), although the attitude holder can shift away from the author in some contexts (see Amaral et al 2007; Harris & Potts 2009; Kaiser 2015; Kaiser & Grosz 2021).

To make this prediction explicit, we propose lexical entries such as (31a) and (31b) for negatively valenced () and positively valenced () emoji, respectively. For concreteness’ sake, these lexical entries build on an indexical presupposition approach (Schlenker 2007; Sauerland 2007), but nothing hinges on this choice. As shown in (31a), we propose that an emoji such as essentially adds a non-at-issue comment on the author’s feeling or attitude towards a proposition p (see Grosz et al. 2023 for a more involved analysis). This proposition p does not need to be directly encoded by the accompanying text, but amounts to a salient proposition retrieved from the context; this is shown in (32a), which intuitively receives the interpretation in (32b).¹⁹ Crucially, the person whose feeling/attitude is conveyed is typically the author (though it is known that the author₀ variable can shift towards another attitude holder, as discussed in the next section).

While the lexical entries in (31–32) are largely in line with Grosz et al. (2023), the present paper differs from Grosz et al. in terms of its aims, empirical scope and theoretical focus. Whereas Grosz et al. look only at face emoji and focus on ‘minimalistic’ positive/negative evaluation (which they gloss as happy/unhappy), the present paper covers both face and activity emoji, distinguishes and analyzes a wider range of emotions, provides a discussion of discourse relations, and argues for the existence of two emoji types that differ in their semantic properties. Thus, the two papers complement each other.

3.2 Shifting perspective

So far, we have argued that the retrieval of an agent/property holder in activity emoji involves anaphoricity, whereas face emoji act like perspective-sensitive expressives when selecting their attitude holder, which exhibit a preference for 1^st-person indexicality. Furthermore, it appears that just like many perspective-sensitive expressions, in the right context, the interpretation of face emoji can shift from the first-person author to another attitude-holder. This was shown experimentally by Kaiser & Grosz (2021) for face emoji in sentences with psych verbs, exemplified in the first row of Table 1.

Table 1

resolution preferences.

	stimulus-experiencer verb	experiencer-stimulus verb
i. attitude holder (face emoji)	richie annoyed adrian	daniel admires aaron
ii. agent (activity emoji)	richie annoyed adrian	daniel admires aaron

Simplifying somewhat, bold type indicates: (i) the preferred resolution of the attitude holder of face emoji, (ii) who the activity emoji provides information about (presumably, who is inferred to be the agent). For a concrete example (“richie annoyed adrian ”), the information should be read as follows: participants preferred the face emoji to be interpreted as expressing Adrian’s emotion, as opposed to Richie’s, or the author’s. This shows that, with face emoji, shifting (here: from the author to Adrian) is possible if the character has the thematic role of experiencer, and both author and character are plausible attitude holders²⁰ (see Kaiser & Grosz 2021 for details). In contrast, activity emoji in Kaiser & Grosz’ experiment replicate the implicit causality patterns found in naturalistic data in (18) and (19), by preferring the stimulus argument (e.g. Richie, Aaron).

In the case of face emoji, Kaiser & Grosz focused on contexts where both characters (as well as the author) are, in principle, plausible attitude holders for the face emoji (e.g. Daniel could be happy because he is admiring Aaron, and/or Aaron could be happy because he is admired). However, what happens if the characters are not plausible attitude holders – in other words, if the emoji is not semantically compatible with them? This is exemplified in (33). Here, emoji users’ introspective judgments suggest the judgments indicated by the prefixes ‘^OK’ (definitely acceptable), ‘??’ (marginally acceptable), ‘?#’ (more or less unacceptable), and ‘#’ (definitely unacceptable). In (33), we see a preference for the emoji to be resolved towards the author of the message. (For completeness, we also provide judgments from emoji users for the same text paired with an activity emoji, which align with Kaiser & Grosz’ experimental results.)

This suggests that when there is no plausible experiencer argument available, the default author orientation (1^st-person indexicality) emerges. An author orientation also shows up in Kaiser & Grosz’ (2021) experiment in contexts with transfer verbs (e.g. brought), which do not have an experiencer argument: Here, face emoji exhibit a significant preference for an author interpretation in examples like (35). This is expected, as this is the default resolution preference for face emoji.

We conclude that the attitude holder of face emoji is often the author, but it can shift to a potential attitude holder mentioned in the text if such an association is plausible, e.g. if the referent is the experiencer of a psych predicate that semantically does not conflict with the emoji’s meaning. These findings are compatible with an analysis of face emoji as expressives, given that expressives have been argued to shift away from the speaker (Amaral et al. 2007), though they do so under highly constrained conditions (Harris & Potts 2009).

3.3 Face emoji and scope

In Section 2.3, we discussed the projective properties of activity emoji and showed that discourse relations play a key role. However, under our analysis, discourse relations are not relevant for face emoji to the same extent, because, as we saw in Section 3.1, they are not integrated with the text via discourse relations, but instead comment directly on the text. In what follows, we analyze the projective properties of face emoji separately from those of activity emoji.

Face emoji are the plausible online counterpart of facial expressions. Taking this as our point of departure, we note that Schlenker (2018: 313–314) argues that co-speech facial expressions, like co-speech gestures, can trigger co-suppositions. In the constructed example in (36) (from Schlenker 2018: 313) a disgusted facial expression co-occurs with the verb phrase skiing with his parents. Schlenker argues that this facial expression falls under the scope of the negation won’t, thereby giving rise to the conditional presupposition that for Sam to go skiing would be disgusting, from either the speaker’s or Sam’s perspective (Schlenker remains neutral on the issue).

Unlike co-speech facial expressions, (36), face emoji seem to be preferably interpreted as scoping over the accompanying text, as shown in (37), which is constructed to be maximally similar to Schlenker’s example. Following our analysis in Section 3.1, we can interpret the nauseated face in the constructed example (37a) as negatively evaluating the entire clause²¹, (37b), therefore scoping outside the negation (see also Schlenker 2018: 344). By contrast, the reading in (37c), in which the emoji takes scope below the negation, seems to be unavailable (though emoji users’ intuitions on the availability of (37c) are far from clear, compared to the uncontroversial availability of (37b)).

A core difference between facial expressions and face emoji has to do with their modality. Facial expressions are in a different modality from speech (one is visual, one mostly auditory) and can thus potentially overlap/co-occur with speech. However, face emoji and text are both written and thus have to be linearized relative to each other (they cannot overlap). It could be this fundamental difference that gives rise to the contrast between (36) and (37).

This is not to say that face emoji are completely unable to give rise to readings of the type observed in (36). While we emphasize that additional careful empirical work is needed to further elucidate these behaviors of face emoji, we review two candidate constructions in the following sections: cases where face emoji may scope below negation when the most salient reading (under which they scope outside negation) is infelicitous and thus disfavored (Section 3.3.1) and cases where emoji repetition is used as a scope-marking device (Section 3.3.2).

4.1 A lexicalist (convention-based) approach

Adopting a term from Maier (2021), lexicalist approaches to the meanings of emoji assume a relatively rich semantics for face emoji, as given in (42) for and .²⁷ Under such a view, the lexical entry of the emoji specifies general properties of the emoji type, such as the presence of an attitude holder (author₀ in (42a-b)), but also more specific aspects of a given emoji, such as the particular emotion that a face emoji encodes (here: angry vs. moderately sad). The analysis in (42a-b) is purely symbolic (convention-based) in that it does not account for the iconic (resemblance-based) nature of emoji; we return to this issue in Section 4.3, after reviewing a purely iconic, picture-based approach in Section 4.2.

Note that there are various ways of fleshing out a symbolic semantics; while (42a-b) uses English emotion terms (angry and moderately sad) in the meta language, the denotations could be defined more abstractly, e.g. in terms of their valence and arousal properties on a numerical scale (see Russell 1980, Jaeger et al. 2019). These issues are beyond the scope and aims of the current paper, but are an important avenue for future work.

While we did not posit concrete lexical entries for activity emoji in Section 2, a symbolic semantics can be given for activity emoji in the same way in which (42) outlines a symbolic semantics for face emoji. To illustrate, the lexical entry of a football emoji may specify a football-playing event, and also include a contextually given agent argument, represented by a contextually assigned variable g(1) in (43a). While (43a) is motivated by the anaphoricity of activity emoji, it is far from clear that all aspects of the rich lexical entry (43a) are part of the emoji’s semantics. The more minimal (43b) assumes that the presence of an agent is pragmatically inferred from the properties of football-playing events. Even more minimally, (43b) may actually derive from (43c) by virtue of event coercion.

It is beyond our scope to adjudicate between the alternatives in (43a-c); a compelling point for (43c) may emerge from the coexistence of “bicycle” () and “person biking” () — if future empirical investigation can demonstrate that they pattern differently along the lines of an object-vs-event distinction, mapping onto (44). (An orthogonal open question is whether denotes the biking activity, (44b), or its agent, as suggested by the emoji’s Unicode name.)

Concluding this section, we wish to emphasize that the difference in how face emoji are modeled as propositional operators, (42), and activity emoji as event descriptions, (43a-b), is not coincidental; it captures the fundamental difference between these emoji types: face emoji operate on a contextually given proposition p, whereas activity emoji denote event descriptions.

4.2 An iconic (picture-based) approach: Maier (2021)

Symbolic approaches of the type outlined in Section 4.1 contrast with radically minimalist iconic approaches to emoji semantics, as proposed by Maier (2021), who argues for an explicitly pictorial account: emoji are simply ‘little pictures’. His analysis of face emoji is summarized in (45); this view does not include reference to the author or to emotional attitudes within the lexical entry. The author-orientation emerges from the assumption that face emoji are interpreted with regards to a viewpoint v_c that typically faces the author of the message. In other words, if an author types ‘it’s raining ’, then the message communicates that the author looks like the picture while typing the message ‘it’s raining’. A pictorial semantics thus assigns more aspects of the meaning to the pragmatics than a lexicalist semantics.

Similarly, Maier proposes an analysis of non-face emoji that is summarized in (46).

In Maier’s approach, the issue of perspective dependence (1^st person indexicality), which we discussed in Section 3, would be tied to the contextually provided indexical viewpoint variable v_c, which would have to face the author by default, but would need to allow for shifting towards a non-author attitude holder in suitable contexts. Similarly, the issue of anaphoricity that we discussed in Section 2 would build on pragmatic enrichment of the semantics in (46) when it involves a picture that connects to an activity or property.

Broadly speaking, a pictorial semantics is extremely minimal and thus requires less assumptions than a lexicalist approach. At the same time, there is ample anecdotal evidence for conventionalization in emoji use, which necessarily goes beyond a purely pictorial interpretation of the emoji: an example is the ‘weary face’ emoji , which has been observed to “convey various feelings of frustration, sadness, amusement, and affection” (Emojipedia).²⁸ Positive interpretations frequently occur in contexts where something is considered cute, (47).

The positive interpretations of the ‘weary face’ do not follow from its resemblance to corresponding facial expressions, which are exclusively negative (see Fugate & Franco 2021).

4.3 A hybrid approach: lexicalism with iconicity

Notably, in the domain of lexicalist approaches, there are at least two options worth considering. Building on Greenberg (2021), we observe that emoji may be symbolic, as discussed in Section 4.1, but emoji may also have convention-based lexical entries that incorporate iconicity. A hybrid lexical entry that incorporates iconicity into a convention-based lexical entry is given in (48) for face emoji, and in (50) for activity emoji.

Developing an iconic semantics in the spirit of Greenberg (2021), we may capture all face emoji with a single lexical entry, as sketched in (48). To unpack this analysis, we arbitrarily pick as a place-holder for all face emoji, i.e. stands for {, , , …}. Greenberg’s (2021) idea is that a lexical entry for a given sign is iconic if the form of the meaning-bearing object also occurs in its denotation. In (48), this means that occurs both to the left and to the right of the iff connective.

If we substitute any face emoji for in (48), we get a well-formed lexical entry, two of which are given in (49a–b). The iconic semantics in (48) is rule-like in that a lexicon of face emoji does not need to store (49a) or (49b); they simply derive from the more general (48). Crucially, such a hybrid approach, which builds iconicity into a lexicalist view, eschews the semantic minimalism of a purely pictorial approach of the type that we saw in Section 4.2. The lexical entries in (48) and (49) still encode the attitude holder, a target proposition p and the presence of an emotional attitude with regards to this p, all of which go beyond a picture of a facial expression.

The present paper does not aim to argue for one approach over the other, and we leave it as an open question whether a lexicalist view has more explanatory power, or a pictorial view, and to what extent iconicity should be incorporated into the meanings of emoji. At this point, our main goal is to highlight the differences between face emoji and activity emoji; we hope that this work can provide a foundation for future work regarding the choice of iconic vs. symbolic approaches to emoji semantics.

Having considered a hybrid lexicalist approach with iconicity for face emoji, (48), a hybrid analysis of activity emoji is given in (50) (with caveats relating to the distinction of object- and activity-depicting emoji, e.g. /, as discussed in Section 4.1).

To exemplify how (50) derives the meanings of different activity emoji, (51) derives from (50) by substituting the football emoji for the place holder.

As with symbolic lexical entries (Section 4.1), it is an empirical question of how minimal or enriched a hybrid lexical entry should be; (51) would correspond to (43b), but generalized lexical entries that generate counterparts of the richer (43a) or the more minimal (43c) are straightforward, as sketched in (52a) and (52b), respectively.

With more minimal lexical entries, (50) and (52b), the existence of a discourse referent associated with the agent of the emoji-based event would be pragmatically inferred, similar to what Maier (2021) assumes in his pictorial approach, whereas it would be semantically encoded in (52a). It is an important empirical question for future research how rich the lexical entries are that emoji users command. Each of the alternatives in Sections 4.1–4.3 are compatible with our proposal.

4.4 Conclusions

In this paper, using a variety of data sources — introspective intuitions, naturalistic Twitter examples, and experimental evidence — we propose a classification of emoji-text combinations, focusing on face emoji and activity emoji. Extending observations from prior linguistic work to a new domain, we show that these two emoji types differ in their semantic properties: face emoji exhibit perspective dependence while activity emoji exhibit discourse anaphoricity. Consequently, these two emoji types are resolved in different ways. Specifically, we analyze face emoji as expressive elements that are anchored to an attitude holder and comment on a proposition in the accompanying text: they share the perspective dependence of natural language expressives, with a preference for first-person indexicality. We analyze activity emoji essentially as event descriptions that serve as separate discourse units (similar to free adjuncts) and connect to the accompanying text by virtue of suitable discourse relations. We provide further evidence for the distinct behavior of face emoji and activity emoji, in particular the role of discourse relations in the case of activity emoji, by looking at their scopal behavior with respect to linguistically-expressed negation. As a whole, this work identifies two semantically distinct subtypes of emoji that interact with linguistic material in distinct ways, and can provide a foundation for future investigations of emoji semantics and the interplay between linguistic expressions and emoji.