3.1.1 Materials

Inspired by previous work, e.g. Hunyadi (2002), suggesting that rich discourse environments may be crucial in allowing participants to make valid scope judgments, we presented the critical sentences together with an introductory context that we considered ambiguous and unbiased as to the relative scopes of the operators in question.

An item consisted of a context description with a particular critical sentence frame. The critical QDP whose scope was being tested was always postverbal, and the other scope bearing element was a constituent in the preverbal focus position (thus, having an exhaustive/identificational reading).

The experimental items were of four types, to be referred to by the letters A, B, C, and D. They differed in the grammatical/thematic status of the critical QDP, as well as in the type of the phrase in the focus position within the critical sentence frame, as described in (14). Each type was represented by 16 items.

There were two experimental factors: the type of the QDP (QuantType) and Prosody. The first factor had four levels: the QDP could contain the determiners minden ‘every’, legalább n ‘at least n’, több mint n ‘more than n’ and sok ‘many’. All DPs containing the determiners above are right upward monotonic, and can appear in the preverbal quantifier position. Among them, DPs with minden ‘every’ can only occupy the quantifier position in the preverbal field; DPs with legalább ‘at least’ and több mint ‘more than’ can additionally appear in the focus position; and those with sok ‘many’ may occupy both the topic and the focus positions in addition. The second factor, Prosody, had two levels: the QDP could be stressed or unstressed. The combination of the two factors resulted in 8 conditions, numbered as shown in Table 3.

Each sentence frame containing a particular QDP was associated with a pair of diagrams intended to depict the two possible readings of the sentence (LS vs. IS), presented side-by-side. The diagrams contained representations of individuals arranged in two columns, and arrows that aimed to indicate which individuals the relation described by the verbs is assumed to hold between. In all diagrams, the first column corresponded to the subject noun phrase and the second column to the object/patient noun phrase. The side of the linear scope interpretation was balanced: half of the items in each set are linear-left, and half are linear-right.

Below we illustrate one experimental item of type A, which consists of a context description, whose English translation is shown in (15), and the sentence frame filled in according to the requirements of the eight conditions in (16)–(19). The pair of diagrams that were used to represent the two readings of the two prosodic variants of the critical sentence in (16) is shown in Figure 1. The two readings of the prosodic variants of the critical sentences in (17)–(19) were represented with identical pairs of diagrams, shown in Figure 2. In these examples, the diagram on the left corresponds to the linear scope reading of the respective sentence and the diagram on the right corresponds to the inverse scope reading. The English translations of oktatók and dolgozatok are professors and papers, respectively.

Figure 1

Diagram pair accompanying the two variants of (16).

Figure 2

Diagram pair accompanying the variants of (17)–(19).

The next examples illustrate an item of type B, in Conditions 1 and 5 (without the introductory context):

(12) above, repeated here as (22), illustrates an item of type C, in Conditions 1 and 2. (23) illustrates the same item in Condition 3:

Figure 3 shows the pair of diagrams used in the trials where variants of (22) appear, where the labels lányok and ruhák stand for girlfriends and dresses, respectively.

Figure 3

Diagram pair accompanying the two variants of (22).

As a comparison of Figures 1, 2 and 3 shows, there was a difference between the diagrams associated with items of types A–B and of types C–D: in the case of the latter, the column representing the set of individuals in the domain introduced by the focus constituent contained names of individuals, whereas in the case of the former the same column only contained numbers that were supposed to stand for different individuals in the domain.

Here we would like to call attention to one feature of the diagrams associated with items of types C–D, which could have led to unwanted effects: whenever there is a non-quantificational DP in focus position, the linear scope reading is true in both situations represented by the diagrams. According to the linear scope reading of (22), Klára is identical to the individual that tried on every dress (which is compatible with the fact that others tried on some but not all dresses, too), thus this reading is true in both situations depicted in Figure 3. The inverse scope reading says that for every dress, Klára is identical to the individual that tried on that dress (thus being incompatible with a situation where any of the dresses were tried on by somebody other than Klára). This reading is only true in the situation represented by the second diagram in Figure 3. As a result, whereas the first diagram is only compatible with the linear scope reading, the second diagram is compatible with both readings. This means that the choice of the second diagram among the two does not necessarily indicate that the speaker is opting for an inverse scope reading. Thus, it is expected that the higher rate of choices of the second diagram does not mean that the participants prefer the inverse scope reading. Moreover, the second diagram in Figure 3 is the simpler one among the two,⁹ thus it is quite likely that when speakers contemplate their choice, they first check whether the simpler diagram matches the reading they have in mind, and if it does, they do not study the other one. We come back to this issue in the course of discussing the results of the experiment.

The fillers used in the experiments consisted of a context description, an unambiguous sentence containing one or two QDPs, and two diagrams, presented side-by-side. The sentences appearing in the fillers realized three types of structure. In nine fillers, a proper name or adverbial in the topic position was followed by a verb and a postverbal QDP, illustrated in (24) below:

In nine other fillers sentences with one preverbal quantificational expression were used, as illustrated in (25):

The critical sentences in the third group of 18 fillers contained two preverbal quantificational expressions: the first one was situated in the topic or quantifier positions, and the second one in another quantifier position or in the preverbal focus position, illustrated in (26):

For fillers, there was a correct and an incorrect diagram choice, and this was also balanced left-right, though fixed for each filler.¹⁰

Eight presentation lists were created according to a latin square design, eight items per condition. Each list consisted of 100 trials, 64 of which were experimental trials and 36 fillers.

Order was a fixed pseudo-random order, such that critical quantifiers were not repeated next to each other, conditions per set were divided evenly between the first half and second half of the experiment, and diagram order was not repeated more than three times in a row.

Four fillers were used as practice items, and the remaining fillers were spread throughout the experiment pseudo-randomly.

3.1.2 Procedure

In Experiments 1 and 2, participants were tested individually, with the help of a laptop computer and headphones. In Experiment 1, the procedure was the following. First, the three-sentence introductory text was displayed on the screen. Participants were asked to press a button after they had read the text, and then were presented with an experimental item or filler aurally. Next, a screen with the two diagrams appeared. Participants were asked to choose the diagram that corresponded better to the meaning of the critical or filler sentence in their opinion, by pressing the appropriate button on the keyboard. On the next screen, they were asked to provide ratings (between “1” and “7”, the latter being the best) as to their confidence in their choice. The intent of the confidence ratings was to help confirm that participants were not confused by the diagrams and simply guessing.

The experiments were put together using the software package DMDX (Forster & Forster 2003), which controlled stimulus presentation and recorded responses.

The procedure in Experiment 2 differed in one respect from the one described above. In this case, participants were asked to listen to the test sentences twice, both before and after seeing the diagrams (in case they were merely “forgetting” how the critical sentence was pronounced). As mentioned above, phonological stress was exaggerated by the recorded speaker in Experiment 2, to rule out the possibility that the stress manipulation in Experiment 1 was simply too subtle to be noticed.

3.1.3 Participants

For the two experiments, participants were recruited from various universities in Budapest (43 and 33 persons, mean ages: 23.6 and 24.3, respectively). They were all native speakers of Hungarian, and had received no previous training in linguistics. They received instructions on paper, and then were tested individually. Testing took approximately 50 minutes, and participants were paid 2500 HUF. Detailed instructions were given at the beginning of the protocol, and participants were allowed to ask questions following the practice items.

3.1.4 Results

The results of the two experiments turned out to be interesting in several ways. First, they contradict the view according to which stress has an effect on the scope of postverbal quantifiers, and second, they indicate that scope judgments might be influenced by other grammatical features whose effects have not yet been investigated for Hungarian.

Figure 4 shows the overall proportion of inverse scope judgments across the two experiments. For both stressed and unstressed postverbal quantifiers, inverse scope judgments were given roughly 25 per cent of the time. This is in stark contrast with those reports in the literature that consider inverse scope judgments to arise exactly in those cases where the postverbal quantifier is stressed. As is also evident from Figure 4, the different prosodic realizations of the stimuli in Experiment 2 had virtually no impact on the results.

Figure 4

Effect of prosody in Experiments 1 and 2.

More interestingly, as Figure 5 indicates, there were some differences in scope judgments across the different postverbal QDPs: those containing the universal determiner minden ‘every’ received more inverse scope judgments than the others. However, the lack of effect of prosody on scope appears to hold across all of the quantifiers investigated. Finally, Figure 6 shows that these patterns are similar across Experiments 1 and 2.

Figure 5

Effects of prosody and quantifier, collapsing Experiments 1 and 2.

Figure 6

Effects of prosody and quantifier, split across experiments.

The overall accuracy rate of fillers was 95% in Experiment 1 and 97% in Experiment 2. Thus, there were no concerns that response patterns may have been a result of participants simply not paying attention, or their inability to decipher the diagram stimuli.

3.1.5 Statistical analysis

We analyzed these results statistically with logistic mixed-effects (Gelman & Hill 2007; Jaeger 2008), using the lme4 package in the statistical software platform R (Bates et al. 2014; R Development Core Team 2011). We fit several models, corresponding to different sets of predictors, predicting the binary response of scope judgment (i.e. 0 = linear scope, 1= inverse scope). The most striking result here is a null result, demonstrating no effect of prosody. However, p-values obtained from standard null-hypothesis significance testing can only inform when a null hypothesis should be rejected, and cannot assess the degree of evidence for a null effect. That is, especially since these results run counter to strong claims by certain authors in the literature, it is not sufficient to merely demonstrate “non-significance” of the effect of prosody. Therefore, we carried out an Akaike weight analysis (see Burnham and Anderson 2002 for a comprehensive review, and Wagenmakers and Farrell 2004 for a brief, accessible tutorial), using the AICcmodavg package for R (Mazerolle 2015). In this type of analysis, model comparisons are made on the basis of the (corrected) Akaike Information Criterion (AICc), a commonly-used model-fit statistic. All candidate models are compared at once, and Akaike weights are computed, which represent the relative strength of evidence for each model as compared to the set of models, and which sum to 1 over the set of candidate models. The ratio of the Akaike weights between different models can be interpreted as relative strength of evidence.

In logistic mixed-effects models, random effects are specified in order to capture the hierarchical (e.g. repeated-measures) nature of the data. Following the recommendations of Barr et al. (2013), we fit random intercepts for subjects and items, but also the “maximal” random slopes for each model. That is, for any within-subject factor, we fit a random slope that allowed the effect to vary by subject, and the same for items. This is arguably the most appropriate way to model standard assumptions about possible between-item and between-subject variation, and the best way to generalize beyond the current set of subjects and items.

The first set of model comparison results is summarized in Table 4. Each row corresponds to a fit model, and marks indicate which factors and interactions were included in that model. The models are given in order from best (top) to worst (bottom), and the corresponding AICc values and Akaike weights are given in the rightmost columns.

The results in Table 4 very clearly rank the model with only QuantType as a factor (Model 1 in the table) as the best model. The next best model has QuantType and Prosody as predictors (without the interaction), but the ratio of Akaike weights between these models is 26-to-1 in favor of the model without Prosody. This indicates that not only is there a lack of evidence for a robust effect of prosody on scope judgments, there is strong evidence against an effect of prosody, such that among the factors investigated here (including prosody and its interactions with quantifier and experiment), a model with only QuantType as an explanatory factor is the far superior model.

For completeness, we also provide a coefficient table for Model 6 in Table 5, which is the model including the effects of Prosody, QuantType, and their interaction. Estimates, standard errors, and p-values are provided for each effect in the model, and on the right side of the table, the standard deviations are provided for the random effects in the model. Note that these random effect standard deviations are on the same scale as the effect estimates, thus giving a sense of how much variability was observed in the effects across items and subjects. The estimates themselves indicate the size of the effect in (log-odds) probability of an inverse scope judgment due to that factor, such that positive coefficients mean a greater probability of inverse scope judgments. Note also that because QuantType is a four-level factor, it must be dummy-coded into three effect coefficients. In the model represented in Table 5, legalább ‘at least’ is used as the “reference” level, and the coefficients represent how the other quantifiers differ from this one. Table 5 confirms what is shown in the graphs above, that inverse scope judgments are significantly more likely for the universal quantifiers, but there is no evidence of effects of prosody.

In contrast to this pattern of null effects for prosody, we found consistent effects of both the grammatical/thematic role of the postverbal quantifier and the type of preverbal operator, such that postverbal subjects (agents) took inverse scope more readily than objects/themes, illustrated in Figure 7, and postverbal quantifiers took inverse scope more easily over a non-quantificational focus than over focused number phrases, shown in Figure 8.

Figure 7

Effect of grammatical/thematic role.

Figure 8

Effect of focus constituent type.

We conducted a similar analysis for these factors as before, applying the Akaike weight analysis for a series of logistic mixed-effects models. For these models, we included QuantType as a factor, but in order to simplify the models, we simplified the QuantType factor to a binary factor, since in the results above, the effect of QuantType reduced to a difference between minden ‘every’ and the others. In order to confirm that this simplification was justified, we carried out a larger set of model comparisons, and all of the models that reduced QuantType to a binary variable (minden ‘every’ vs. the others) were superior to models which kept them all separate. For reasons of space, we do not give the full details here, but the Akaike weight evidence ratio was well over 1000-to-1 for the binary-factor models, indicating very clear results. Therefore, in the following analyses we collapse the QuantType factor into a binary factor. Additionally, since no effects of Experiment or Prosody were observed above, we do not examine those factors here. The table of models is given in Table 6, along with the corresponding AICc values and Akaike weights. Since our items were constructed such that all grammatical subjects were also agents, we simply term this factor “Role.”

The results in Table 6 suggest that Role and FocType both play a role in affecting scope judgments, and furthermore that there is some interaction between these effects and the QuantType. The model coefficient table for the best model (Model 1 in Table 6) is given in Table 7. The statistical results confirm what is apparent in Figures 7 and 8, namely that there are clear effects of Role and FocType when the determiner of the critical postverbal QDP is minden ‘every,’ but with the other quantifiers, the effect is smaller for Role and not apparent at all for FocType.

4.1.1 Materials

In Experiment 3, the same experimental items and fillers were used as in Experiments 1 and 2, including the 3-sentence texts setting up the context. The audio files were identical to those used in Experiment 1. For each item in each condition, we created an unambiguous paraphrase of the inverse scope reading that consisted of two preverbal QDPs, the scope of which is assumed by all theoretical accounts in the literature to be unambiguous. (27-a)–(27-d) illustrate the unambiguous¹³ paraphrases provided for (16)–(19) above:

In response to concerns about participants being fatigued by the length of Experiments 1 and 2, we took the same experimental lists that we used in the previous experiments, but only used the first half of each. This resulted in having 32 experimental trials (four in each condition) and 16 fillers in each list.

4.1.2 Procedure

The participants were tested again individually with the help of a computer. First, the three-sentence context-description appeared on the screen. Next, the critical sentence was presented aurally. After this, the written paraphrase of the inverse scope reading appeared on the screen. On the next screen, the participants were asked to indicate on a five-point scale how well they thought the critical sentence matched the meaning of the (unambiguous) sentence presented in writing. (“1” stood for minimal and “5” for a great deal of similarity.)

4.1.3 Participants

Thirty-three participants (mean age: 27.2) without previous training in linguistics took part in the experiment, recruited again from universities in Budapest. They received 1500 HUF for their participation.

4.1.4 Results and statistical analysis

In order to simplify the discussion and analysis of results, we dichotomized the ratings such that responses of “1” or “2” were coded as “linear scope” judgments and responses of “4” or “5” as “inverse scope” judgments. Ratings of “3” were taken to be ambiguous or undecided, and were excluded from the statistical analysis. This decision is justified in part by the fact that the distributions of ratings peaked at “1” and “5”, indicating that the judgments were essentially binary judgments with some uncertainty superimposed. The patterns shown in the dichotomized results are the same as revealed by the full ordinal scale, and are easier to read and interpret. Therefore in the statistical analysis, we report the results of logistic mixed-effects models, comparable to the models fit for Experiments 1 and 2. We also fit ordinal mixed-effects models (using the ordinal package, Christensen 2015), which showed the same pattern of results as the logistic models reported below. But again, because the dichotomized results are easier to follow and interpret, we present only those here. Likewise, we focus on the binary QuantType variable, comparing the pattern of results for DPs with minden ‘every’ vs. the rest of the quantificational expressions, following the approach taken in the case of Experiments 1 and 2.

First we consider the effect of prosody on scope readings. Figure 10 shows the pattern of judgments, broken down by quantifier. If prosody had the effect reported in É. Kiss’s and Hunyadi’s works, the heights of the darker bars should be close to 1.00 (“inverse” judgment), and the heights of the lighter bars should be close to 0.00 (“linear” judgment). As Figure 10 shows, still no effect of prosody is observed. The only (non-significant) numerical trend is for minden ‘every’, in the opposite direction than expected.

Figure 10

Effect of prosody in Experiment 3.

The results of the model comparison analysis are given in Table 8, and the model coefficients for the full prosody by quantifier interaction (Model 5 in Table 8) in Table 9. Note that Experiment is no longer a factor because we are examining only the results of Experiment 3. These results unambiguously replicate the pattern of results from the first two experiments. Notably, the overall rates of inverse scope judgments are much higher than in Experiments 1 and 2. As in Experiments 1 and 2, inverse judgments are more acceptable with postverbal DPs containing the determiner minden ‘every’, though the effect is milder, and just shy of p < .05 significance (see Table 9). And just as in the other experiments, the Akaike weight analysis indicates strong evidence for a null effect of prosody. The model with only QuantType as a predictor is a much better model than a model with both QuantType and Prosody, with a 55-to-1 evidence ratio.

Experiment 3 also replicated the pattern of results regarding the effects of Role and FocType. Figures 11 and 12 show that postverbal subjects take inverse scope more easily, and that inverse scope is more acceptable when the constituent in focus position is a non-quantificational DP. Interestingly, in this experiment, these effects are evident across the different quantifiers, and not exaggerated for minden ‘every’ DPs.

Figure 11

Effect of grammatical/thematic role.

Figure 12

Effect of focus type.

The statistical analysis also reflects this pattern. The model comparison results are given in Table 10. The two best models are the model with only simple effects of QuantType, Role, and FocType, and the model with the addition of the QuantType × Role interaction. The Akaike weights are not strong enough for either of these to decide between them. However, the evidence is extremely strong for the top five models together, which all have in common the effects of QuantType, Role, and FocType. Thus, we cannot make strong claims as to the presence of possible interactions between these, but clearly these factors are important for explaining the data. As mentioned above, Experiment 3 used only half the items as Experiment 1 and 2. The cost of this reduction is a loss in power to be able to clearly test the various potential interactions between these factors. However, the simple effects of the factors are still very evident.

In order to more easily compare these results to the previous results, we give the coefficients and random effects for Model 3 in Table 11, which corresponds to the same model that is given above in Table 7. The differences between these corroborate what we see in the figures, namely, that the effects of Role and FocType in Experiment 3 appear to be very consistent across types of postverbal quantifiers.