2.1 Cognitive-conceptual approaches to prepositional semantics

The interest in prepositional semantics in Cognitive Linguistics stems from the observation that language users are able to use a relatively small set of prepositions refer to an indefinitely large number of relations and scenes because of their cognitive ability to categorize concepts schematically (Kreitzer 1997). The cognitive-conceptual approach to prepositional semantics relies on two important constructs: the so-called “embodiment” of meaning, i.e. “[t]he idea that the properties of certain categories are a consequence of the nature of human biological capacities and of the experience of functioning in a physical and social environment” (Lakoff 1987: 12), and the notion of image schemas, which can be considered as condensed, schematic, recurring patterns of perceptual experience (e.g. Oakley 2010; Gibbs et al. 1994).

Some key publications in developing the notion of image schemas and embodiment, and integrating those notions into the discussion of meaning representation, are Brugman (1988) and Lakoff (1987). In their analyses of over, Brugman and Lakoff distinguish a vast number of distinct image schemas, all of which map onto a distinct ‘sense’ of the preposition. The image schema underlying an example such as Devi lives over the hill, for instance, conveys a static horizontal spatial configuration, whereby the focal point or “trajector” (TR), Devi, is positioned on the other side of the “landmark” (LM), the hill. This schema is different from the one underlying an example such as The helicopter hung over the hill, which conveys a vertical spatial configuration in which the helicopter (TR) is positioned above the hill (LM). The schema furthermore differs from those underlying examples such as Devi walks over the hill or The helicopter flies over (the hill), which involve a (horizontal) path, and so on.

Yet, while they evoke different spatial configurations, the image schemas underlying these examples are still connected to one another, as humans are able to recognize general similarities between abstract image schemas (as demonstrated experimentally by, for instance, Gibbs et al. (1994)). Thus, a complex yet structured ‘network’ of linked senses is formed. Such networks, often termed ‘lexical networks’ or ‘polysemy networks’, comprise of nodes which are situated at varying distances from one another, and are centered around a primary sense or prototype (Lakoff 1987; Rice 1996). At their core, prepositions are spatial expressions, but the general human ability to apply metaphorical and analogical reasoning allows them to extend the use of prepositions to embody the non-physical domain ubiquitously (Kreitzer 1997: 317; Lee 1998: 334; Rice 1996: 135; Rice 1999: 227), as demonstrated by examples such as Devi works over the weekend (embodiment of time) and Devi is over her ex-boyfriend (embodiment of mental state).

As each small modification to an image schema is mapped onto a discrete sense category, the meticulous and comprehensive accounts set out by Brugman and Lakoff are sometimes called the “full-specification” approach. In the case of Lakoff (1987), the full-specification approach led to a fine-grained overview of 24 senses of over, which are connected in a sizable polysemy network. In later work, Lakoff’s proposal was criticized amply for the fact that it leads to a virtually unconstrained number of sense categories, and for lacking methodological rigour (e.g. Rice 1996; Kreitzer 1997; Sandra & Rice 1995; Tyler & Evans 2001; 2003). This led Sandra (1998: 361) to coin the term “polysemy fallacy” in reference to “the tendency to look for polysemy even when there is no evidence for it”.

Subsequent proposals, then, set out ways to tackle the apparent lack of a principled procedure to determine the number of distinct (sub)senses. Two notable examples are the proposal of Kreitzer (1997), and the “principled polysemy” approach advocated by Tyler & Evans (2001; 2003). Drawing strongly on the spatial information encoded in linguistic expression, Kreitzer (1997: 308) defines a prepositional sense as “a class of uses sharing a unique relational level image schema”. In the case of over, Kreitzer argues that only three such schemata can be distinguished: (1) a static relation between two points on a vertical axis (over₁), (2) a dynamic relation involving a path schema (over₂), and (3) a static relation where one point occludes the other (over₃):

These three relational schemata are also applicable to non-spatial domains, which, Kreitzer explains, are consistently conceptualized in terms of spatial image schemata: the use of over in I finally got over that relationship (indicating a path obstructed by an obstacle), for instance, can be motivated by the dynamic schema underlying over₂, whereas over in The box is over six feet tall (indicating excess) is motivated by over₁. Yet, as pointed out by Tyler & Evans (2001: 729), Kreitzer does not motivate the existence of those three relational image schemas in light of each other, as he “makes no attempt to account for how over₁ could give rise to over₂ and over₃ respectively”. Additionally, many senses touched on by Lakoff (1987) are simply ignored in Kreitzer’s account.

Addressing these issues, Tyler & Evans (2001; 2003) devised a more encompassing proposal based on slightly different principles. More specifically, they argue that senses can be considered distinct if (and only if) under the following criteria: (i) First, assuming that the primary sense of the preposition involves “a particular spatial relation between a TR and an LM”, the distinct sense “must involve a meaning that is not purely spatial in nature”, or “the spatial configuration between the TR and LM is changed vis-a-vis the other senses associated with a particular preposition” (Tyler & Evans 2001: 731). (ii) Second, the sense must exist in examples where it cannot be inferred from the combination of another sense and encyclopedic or contextual knowledge (i.e., they must be instantiated in semantic memory; Evans (2005)). Following these criteria, there is no reason to assume that the use of over involves two distinct and separately stored senses in The helicopter hovered over the hill and The helicopter flew over the hill, as both examples involve a spatial relation in which the TR (the helicopter) is located above the LM (the hill). Furthermore, the difference in stativity/dynamicity of the scene can simply be inferred from the lexical verb (hover vs. fly). As such, Tyler and Evans effectively constrain the full-specification network to a more digestible size.

An issue that remains, however, is that there is still no objective, measurable means of determining the global structure of the polysemy network. Besides further attempts to establish which sense constitutes the core node or prototype (Sandra & Rice 1995; Rice 1996; Gilquin & McMichael 2018; Newman 2011: 137), there is still much room for discussion regarding the position of derived nodes. To illustrate the issue, we can consider the multitude of possible derivation pathways of the repetitive sense of over, as in She sang the same song over (and over) (Tyler & Evans 2003: 105–106). First, based on their comparable, cyclical image schemas, the repetitive sense can be connected to reflexive uses of over (e.g. She turned the page over/The vase tipped over). Second, it is possible that repetitive over marks an iterative trajectory, in which case the sense could be derived from cases where over marks the end of a linear temporal trajectory or process (e.g. The race is over). A third possibility is that the repetitive sense constitutes a conceptual blend of reflexivity and trajectory completion, a notion which may equally apply to many other derived senses.

In their accounts, Tyler & Evans choose to remain agnostic on the matter, explaining that “language does not function like a logical calculus which would allow us to … establish absolutely a single, precise derivation for each sense” (Tyler & Evans 2003: 62). This is not to say that ‘anything goes’, but rather that there is a delimited set of general principles or paths of derivation which may individually or simultaneously give rise to derived senses, and different individuals may draw different connections between senses, if they draw any such connections at all (Langacker 2010: Ch.10). Yet, even so, the agnostic position is somewhat unsatisfactory if one is interested in, for instance, comparing the general probability of multiple derivational paths across individuals, or even across time. Such queries will remain difficult to address in absence of methods that enable researchers to quantify and measure the degree of similarity between sense categories. The further integration of distributional semantic models into Cognitive Linguistics, then, can at least partially be linked to the research community’s growing desire to approach the study of polysemy (and synonymy) in a more rigidly corpus-driven and measurable way (Gries & Divjak 2009; Newman 2011).

2.2 Advances in Distributional Semantic Models

At its core, the distributional approach conceptualizes the meaning of a word (or, more generally, of constructions) as a function of its lexical and grammatical context, and as such, meaning can be approached statistically (Turney & Pantel 2010; Boleda 2020). Statistical approaches to meaning have a long tradition in corpus linguistics, with functional-semantic classifications into distinct usages being increasingly based on explicit, automatically detectable contextual cues as corpora grew increasingly large.

An interesting observation made by Heylen et al. (2015) concerns the statistical-manual hybridity of the corpus linguistic tradition. As an example, they take the “British tradition in corpus linguistics”, in which lexical collocations and syntactic patterns are employed to capture or approximate word meaning, while the classification of these meanings into categories is conducted manually. By contrast, the more recent application of “Behavioral Profiles” (Gries 2006; Gries & Divjak 2009), for instance, presents a means of statistically automating the classification by means of hierarchical cluster analysis (or correspondence analysis in Glynn (2010)). In such cases, a set of tokens can be annotated along a number of variables or dimensions, such as the type of trajector (TR), landmark (LM), as illustrated in Table 1 from Newman (2011).

Such data frames can subsequently be converted into a table with numeric infomation (e.g. the relative frequency of each example with each label), which can then be used as input for statistical analysis (Gries & Otani 2010). Focusing only on Trajector-Landmark combinations found in the ICE-GB corpus, Newman (2011) ultimately identifies seven statistically distinct uses of over by means of a Hierarchical Configural Frequency Analysis (HCFA):

The appeal of this approach, according to Newman (2011: 541–542), is that it “offers a systematic corpus-based procedure”, which is “strongly grounded in facts of usage, complementing any other (intuition-based or experimentally based) methods the researcher might employ”.

Still, the selection and annotation of the variables (which have been selected and defined by the analyst) is predominantly manual. As a “logical extension of the statistical state-of-art” (Heylen et al. 2015: 154), then, Semantic Vector Space Models were introduced, in which all aspects of semantic analysis are approached statistically. In such models, the contextual properties that are fed into statistical classification models are no longer manually annotated features, but automatically generated numeric representations of syntactic and lexical co-occurence patterns.

Because the distributional approach to meaning is based on a relatively simple, and concrete premise, one may be tempted to assume that studies adopting this approach are highly comparable, if not identical in how they operationalize and model meaning. This would, however, be a mistaken assumption. It would be far beyond the scope of the present paper to survey the many different ways in which ‘meaning as context’ has been operationalized (for such a survey, one may consult Turney & Pantel (2010), Lenci (2018), Boleda (2020), or specifically for deep learning based models, Young et al. (2018)). However, to clarify the model choice in the present study, a brief discussion of two relatively recent developments is warranted. This concerns (i) the rise of models operating with contextualized (or, rather, token-based) semantic vectors, and (ii) the rise of context-predicting models (also known as ‘neural language models’) that create semantic vectors often referred to as ‘embeddings’.

2.3 BERT

In the present study, the distributional approach is represented by a single model architecture: Devlin et al. (2019)’s Bidirectional Encoder Representations from Transformers (BERT). First launched in November 2018, BERT quickly became the model to beat due to its impressive performance on a wide range of NLP tasks. Soon after, it also grabbed the attention of computational linguists, who were interested in determining precisely what kind of linguistic information such models acquire and capture (Clark et al. 2019; Jawahar et al. 2019; Alishahi et al. 2019).

In a nutshell, BERT is a deep contextualized model based on a particular type of neural architecture, called “the Transformer”, which is entirely based on so-called “attention mechanisms” (Vaswani et al. 2017). A context-predicting model, BERT has been pre-trained on approximately 3.3 billion words (800 million words taken from the BooksCorpus, and 2.5 billion words from English Wikipedia) of unlabelled data over a masked word prediction task (in which the objective is to predict randomly masked input tokens based only on the context in which they occur) and a next sentence prediction task (so that the model will also understand sentence relationships).

Like other Transformers, BERT consists of multiple layers (or ‘transformer blocks’), all of which contain multiple self-attention heads which behave similarly within their layer. The smallest pre-trained model, called BERT_base, consists of 12 layers with 12 attention heads, whereas the larger model, called BERT_large, consists of 24 layers with 16 attention heads. Each of these layers captures the n tokens in the input sentence (or rather ‘sequence’, as the input need not correspond with what linguists have traditionally defined as a sentence) in compressed numerical vector representations or ‘embeddings’.

The attention heads within BERT’s layers have been probed for the linguistic phenomena they capture. This revealed that particular heads capture syntactic relations (e.g. valency patterns and dependency relations), while others perform well at coreference resolution (Clark et al. 2019) – which is remarkable given that the model has not received any explicit input about syntax or coreference. This “syntax-aware attention” (Clark et al. 2019) may be why BERT is succesful the downstream NLP tasks it has been employed in (cf. Peters et al. 2018b). Finally, it is important to note that the different layers (and accompanying attention heads) perform slightly differently on different tasks. In various sources, the second-to-last layer (or a concatenation of the last four layers) is suggested to perform best on token-level tasks such as word sense disambiguation (e.g. Devlin et al. 2019; Wiedemann et al. 2019), but many applications also operate with the final hidden layer (e.g. Huang et al. 2019; Blevins & Zettlemoyer 2020).

With respect to linguistic investigation into polysemy and sense disambiguation, the contextualized embeddings produced by BERT have thus far not been explored. One reason may be that neural models have grown into increasingly intransparant systems (Linzen et al. 2019: iii), making linguists more reluctant to rely on them for lingusitic analysis. Still, it is worth investigating to what extent they could be employed as analytic tools in linguistic research, as neural language models like (but consistently outperformed by) BERT have already been shown to capture very nuanced aspects of meaning, and they even seem to provide usable representations for function words (see Boleda (2020: 18), in reference to Peters et al. (2018a)). Furthermore, a model such as BERT also unites the strengths of different types of token-based distributional methods. First, the fact that the model is syntax-aware agrees with the cognitive-linguistic (and constructionist) view that differences in syntactic structures reflect differences in meaning (Langacker 1991; Goldberg 1995). As such, its syntax-awareness sets BERT apart from bag-of-words approaches to contextualized vectors (e.g. Heylen et al. 2015), and thus makes it more akin to, for example, the Behavioural Profiles approach. Second, the application of BERT to the study of polysemy does not involve any manual annotation, and neither does it involve making an a priori selection of syntactic features to be included, making it a fully data-driven approach to the question at hand.

4.1 Sense Distinctions

As a first point of enquiry, it is investigated whether BERT indeed recognises the sense categories proposed in the principled polysemy model in a relatively distinct and coherent manner. Following Kilgarriff (2003: 108), I define ‘senses’ as “abstractions over clusters of word usages”. In other words, if the abstract, conceptual sense categories proposed in the principled polysemy model are recognized by BERT, we would expect to find that the geometrical distance (operationalized as the cosine distance) between the embeddings of all tokens labelled as Sense 1, for instance, is shorter than the distance between those tokens and tokens with a different category label, thus forming a cluster.

To visualize the local embedding clusters and the global positioning of those clusters relative to one another, a two-dimensional representation of the token embeddings was created based on the t-Distributed Stochastic Neighbor Embedding (t-SNE) algorithm for dimensionality reduction of high-dimensional data (van der Maaten & Hinton 2008). In Figure 3, each token is represented by a dot, which has been coloured according to its manually assigned label. Note that the embeddings were created solely based on the contextual information surrounding over, and that the manual labels were assigned separately. The distributional model was hence not fed any human-defined knowledge about the number or nature of labelled sense categories. The two-dimensional plot below therefore visualizes the overall correspondence between clustered token embeddings, which can be conceptualized as ‘distributionally defined senses’, and the proposed ‘conceptually defined’ sense labels.

Figure 3

t-SNE embeddings of over, perplexity = 20, KL divergence after 1,000 iterations: 0.457.

From eyeballing Figure 3, it looks like the distributional model proposes a fair number of distinct token-clusters that correspond relatively well with the suggested sense categories: in the majority of cases, the local clusters (or cluster areas) consist of tokens that were assigned the same label. At the bottom of Figure 3, for instance, a local cluster area was marked, which comprises entirely of examples of Sense 2A (more specifically, those cases where the TR has mentally overcome an obstacle or past relationship). Yet, at the same time, the correspondence between the two models seems to become weaker at the global level. In some cases, such as Sense 2C (‘completion’), 2E (‘time span’), 4 (‘examining’), 7 (‘means of communication’) and 8 (‘hangover’), it appears that all tokens with the same label are assigned to the same local cluster area, but in others, such as 2D (‘transfer’), tokens are grouped in separate, relatively distant areas. Thus, the question arises how we can assess the degree to which there is correspondence between the distributionally defined and conceptually defined senses.

To address this question in a way that goes beyond eyeballing a visualization, this study uses a series of classification tasks, which help quantify the extent to which there is correspondence between the sense categories emerging from the two models (VAM). Figure 4 presents the results of the VAM when applied to all categories (over 100 iterations). The x-axis represents the abstraction continuum, which starts at no abstraction (the exemplar level, where classification of unseen tokens (20% of the data) is attempted by means of the nearest neighbour embeddings of concrete tokens) and reaches up to the target level (the highest level of abstraction, where all items within the 16 labelled categories are clustered into averaged ‘sense embeddings’). The y-axis represents the classification accuracy of the distributional model (F₁-score, i.e. the harmonic mean of precision and recall when applied to the test set).

Figure 4

VAM output over all data.

At the lowest level of abstraction, the model’s classification accuracy is quite high at 0.95, remaining relatively stable at this level until approximately 200 tokens have been clustered. Subsequently, its accuracy gradually drops to 0.8 (at approx. 300 tokens), after which it drops below 0.7 at the highest levels of abstraction. These findings imply that the BERT embeddings do encode the similarities between members of the categories proposed in the principled polysemy model, but only up to a certain point. Beyond that point, the proposed abstractions no longer optimally fit the output of the distributional model.

When we assess the models classification accuracy per sense category (Figure 5), we also find that the model is more successful in ‘recognizing’ some sense abstractions than others. Overall, the embeddings of over clearly encode the similarities between concrete tokens, and the performance of the model remains high at lower-intermediate levels of abstraction where only some of the concrete token embeddings are merged into slightly more schematic representations. Yet, whether higher levels of abstraction are still meaningfully encoded in the BERT embeddings of over seems to depend on the sense category under scrutiny. Unsurprisingly, perhaps, it is precisely the sense categories that occur in fixed syntactic configurations and have clear collocational preferences (e.g. Sense 2C ‘completion’, in which over consistently functions as an adprep in combination with a form of BE), are easier to group than more ‘schematic’ senses.

Figure 5

VAM output per sense category.

There are, however, a number of notable cases where the model’s classification accuracy drops sharply when more more items of the same category are merged. This pertains to Sense 3, where the model does not recognize similarities between, for instance, Spread a tablecloth over the table and He received votes from all over the floor. Reassuringly, this ties in with other analyses that have argued that ‘covering’ does not adequately capture the interpretation of all over (Queller 2001; Taylor 2006; Pawelec 2010: 98–101). Furthermore, Sense 4A (‘focus of attention’) and 5B (‘control’) suffer from the high number of false positives of the other category the model wishes to assign to them (see Section 3.3.4–3.3.5 on the difficulty of distinguishing 4A and 5B). Finally, drops in performance can also be witnessed for category 2A and 2D, as the model seems to have difficulties in relating examples describing physical and non-physical scenes.

4.2 Relations between senses

Having established that there is some correspondence between clustered BERT embeddings and the proposed sense categories (up to a certain point), we can now turn to the question whether the geometrical distances between the various senses of over (as emergent from the distributional semantic model) correspond with the semantic relationships proposed in the prepositional polysemy network of over proposed by Tyler & Evans (2001: 746), reproduced in Figure 6.

Figure 6

Polysemy Network of over as presented in Tyler & Evans (2001: 746).

The dark, full nodes in the suggested network representation in Figure 6 constitute what are considered to be separate senses, whereas the empty nodes are included as abstractions over a proposed cluster of related, derived senses. At the centre, we find the protoscene (Sense 1). Because the representation in Tyler & Evans has been constructed based on theoretical principles, it makes little sense to assess the representation based on the absolute geometrical distances between embeddings derived from the distributional model. It does make sense, however, to operationalize the directness of node linkage in the proposed network to relative distances between embeddings. More specifically, we could hypothesize that, if Sense 4A is not directly derived from the protoscene (Sense 1), but emerged as a further extension derived from Sense 4, we would expect that the relative distance between Sense 4A and Sense 1 is bigger than the relative distance between Sense 4A and Sense 4. Similarly, if Sense 2A, 2B, 2C and 2D form a cluster of related senses, we would expect the relative distance between, for instance, Sense 2A and 2B or 2A and 2D, to be shorter than the relative distance between Sense 2A and Sense 3.

To compare the principled polysemy network to the output of the distributional model by means of cosine distances, two approaches can be taken. First, we may approach the comparison by taking the senses proposed by Tyler & Evans (2001; 2003) as given, and rely on manually assigned labels to create an averaged sense embedding – that is, a summary embedding similar to the clusters created at the highest level of abstraction in the VAM. Subsequently, we can calculate cosine similarities between these sense embeddings. If the result of this assessment turns out to be that the relative distance between the sense embeddings maps onto the suggested relative distances in the network in Figure 6, we could conclude that both models arrive at the same network representation in a relatively straightforward manner.

However, as explained in Section 4.1, the proposed sense categories do not always correspond with the way in which the tokens of a manually labelled category cluster, with categories such as 2A falling apart into multiple rather distinct groupings. A second approach, then, would be to adhere less strictly to the sense categories proposed by Tyler & Evans, and determine the geometrical distance between token clusters proposed by the distributional model. In what follows, I restrict myself to this second approach.

Figure 7 presents a hierarchical cluster tree of the annotated tokens. Of the 808 annotated examples, 26 were examples were excluded because they were considered unclear or ambiguous between multiple readings (cf. example (42)). The clustering presented in Figure 7 is based on the cosine distance between the embeddings of the remaining 782 examples. The coloured areas represent clusters of neighbouring tokens that were assigned to the same category. In order for a group of tokens to be considered a cluster, it was decided that there should be at least 5 neighbouring tokens of the same sense category. As such, the smallest clusters represented in Figure 7 are based on at least 5 examples, and the largest (i.e. cluster 2E, ‘time span’) contains 51 examples. In total, 47 examples did not have at least 4 neighbours of the same type.

Figure 7

Cluster tree (distance = cosine) with representative examples.

As was briefly pointed out in Section 4.1, seven of the conceptual categories are ‘consistent’, with all tokens clustering together. This includes 2E (‘time span’), 2C (‘completion’), 4 (‘examining’), 5B (‘control’), 6 (‘reflexive’), 7 (‘communication channel’) and 8 (‘hangover’). For the remaining nine categories, the model suggests that there are at least two different clusters. In some cases, the separate clusters are still part of the same higher-order branch (e.g. 1 ‘above’, 4A ‘focus-of-attention’, 6A ‘repetition’), whereas for others, the embeddings are less closely related (e.g. 2A ‘other-side’, 2D ‘transfer’, 5A ‘excess’). All in all, the suggested distances between the sense groupings differ substantially from the proposal put forward by Tyler & Evans: not only does the distributional distributional model suggest fairly large distances between token groupings that Tyler & Evans would have assigned to the same category (based on their shared underlying image schema), the proposed relative distances in the sense network are also not reflected in the geometrical distances between the (groupings) of embeddings.

Given that BERT does not group tokens of over according to abstract similarities in spatial configurations between them, the question that remains is what kind of groupings the model does suggest, and whether any (other) meaningful abstractions can be made. To address this question, we could examine the token groupings illustrated in Figure 7.

GROUP 1 – SPACE The first group that appears to be recognized appears to be a collection of spatial uses of over. Tokens grouped in Group 1 include all tokens assigned to Sense 1 (‘above’), and some tokens of the spatial ‘excess’ sense, 2B, in which a spatial border or threshold is exceeded. Note, however, that this cluster does not correspond with Kreitzer (1997)’s static over₁: Group 1 also includes a subgroup of tokens of Sense 3 (‘covering’, over₃), (stative) uses of Sense 2A (‘other-side-of’, over₂), and examples involving dynamic verbs (e.g. the cat jumped over the fence, over₂).

GROUP 2 – TIME SPAN and GROUP 3 – COMPLETION The two temporal uses of ‘over’, 2C (‘completion’) and 2E (‘time span’), constitute fairly separate categories. The coherent cluster of 2E tokens will henceforth be referred to as Group 2. The coherent cluster of 2C will henceforth be referred to as Group 3. In all examples in Group 2, over functions as a preposition, whereas in all examples in Group 3 function as an adprep.

GROUP 4 – MIND AND PERCEPTION As the closest neighbour of Group 2, we can discern a very large grouping of tokens where over consistently marks a non-spatial relation (Group 4). In Group 4, we find all examples categorized as 4A (‘focus of attention’). One cluster of examples of Sense 4A involves verbs such as preside and watch. These are closely associated with examples of 2D, in which there is a transfer of power or authority (e.g. He took over the business). These are, in turn, closely related to 5B (‘control’). The other token cluster of 4A (‘focus-of-attention’), which includes all examples where over can be paraphrased with ‘about’ or ‘because of’ (e.g. He agonized over it), is most closely linked to the subgroup of tokens in category 2A, where the TR has mentally overcome or lost interest in the LM (e.g. I am over the drama). We furthermore find all tokens of 5C (‘preference’) in Group 4, closely positioned next to a subgroup of 2B, where the LM is omitted or skipped in a selection procedure (possibly implying absence of preference, e.g. She was passed over for the job). Group 4 also contains all tokens of 4 (‘examining’). Given that the latter group of examples frequently (but not exclusively) involves the phrasal verb combination look over, it is not surprising that these tokens are positioned relatively close to (but, notably, are not confused with) cases where a glance is cast (classified as 2A). Finally, we also find all tokens of Sense 7 (‘means of communication’) in Group 4. While perhaps more loosely related to the ‘mind’ and ‘perception’ relations, Sense 7 also involves animate TRs and a non-spatial interpretation (i.e. in a sentence like they spoke over the phone, the preposition does not capture a physical, spatial positioning of the TR and LM).

GROUP 5 – PATH Related to Group 3, we find a cluster of tokens where over again has a spatial interpretation. Yet, unlike the tokens in Group 1, over functions as an adprep, and involves movement along a path (and hence partially overlaps with Kreitzer (1997)’s dynamic over₂). These include examples of 2D (‘transfer’) as well as examples of 2A where the TR moves to a different location (e.g. I made my way over (to the computer)).

GROUP 6 – EXCESS (IN NUMBERS) In the remaining set of tokens, then, a clustering of 5A tokens can be distinguished, where the LM is an amount or quantitative threshold (e.g. She mentioned that I donated over $100,000 to Katrina victims (2006, COHA)) or an amount (this cluster is also identified by Newman 2011).

GROUP 7 – NON-SPATIAL ADPREPS AND FIXED PHRASES While it is not easy to make sense of Group 7, it is interesting to note that all tokens classified as Sense 6 (‘reflexive’) and 6A (‘repetition’) are part of this group. Yet, rather than clustering together, they are positioned closely to token groupings classified as Sense 3 (in the fixed combination all over, e.g. I feel pain all over), Sense 5A (non-literal adprep uses, as in their personal involvement will spill over into their workplace interaction), and Sense 8 (‘hangover’). The relation between these groupings seems formal rather than semantic, as the majority of groupings present cases where over functions as an adprep, and occurs in relatively fixed phrases.

If we approach the complex internal semantic structure of over by means of BERT embeddings, then, it appears that global clusters are formed based on in intersection of similarities in, on the one hand, conceptual domain (i.e., spatial, temporal, mental, etc.), and syntactic resemblance on the other. Overall, the lack of correspondence between the suggested network configuration in Figure 6 and the global distances between the grouped embeddings does not necessarily indicate that the global groupings are not interpretable, or that no abstractions can be made – rather, it suggests that the use of an embedding-based approach leads to different abstractions.