2.1 The cluster factor

For both underlying and epenthetic schwa, schwa is realized more often after two or more consonants than after a singleton consonant. The examples in (10) show this for deletion, while controlling for phrase position. In all examples, schwa is also followed by a consonant, since schwa is rarely realized adjacent to a vowel.

The number of preceding consonants also plays a role in epenthesis, as shown in (11).

Support for the probability judgments above are found in our experimental results, previewed in Table 2. The data show that across rhythmic and morphological contexts, schwa is realized more often in the context CC_ than in the context C_. These results also show that schwa is generally realized more often when it’s underlying (at a clitic boundary) than when it’s epenthetic (at a word boundary), as described in the previous section.

In the constraint-based models that follow, we model the cluster factor with the constraint *CCC, which militates against a sequence of three consonants (Grammont 1914).

Under the *CCC analysis, schwa is inserted in phrases such as la secte partait [la sɛktœpaʁtɛ] because schwaless [la sɛktpaʁtɛ] contains the cluster [ktp]. Similar constraints have been used in previous analyses of French schwa, such as Côté’s (2000; 2007) constraint *C⟷V, which requires every consonant to be next to a vowel, and Kaplan’s (2011) constraint *NTN, which militates against stops flanked by non-approximants. For the data we model, *NTN and *CCC are interchangeable, since all of the CCC sequences in our data set contain a stop as the second consonant in the cluster.

2.2 The stress factor

An effect of phrase position on the probability of realizing schwa has been observed at least since Léon (1966), who describes schwa as more likely to be realized in the penultimate syllable (see also Morin 1974; Dell 1985; Tranel 1987). The examples below show this for schwa at word boundaries (13) and clitic boundaries (14).

In both (13) and (14), schwa occurs after two consonants in the context VCC_C. In the context VC_C, it has been claimed that there is no effect of the number of following syllables, regardless of whether the schwa is at a word boundary or clitic boundary. Côté and Morrison (2007: 169) report that the probability of schwa realization is the same in (15a) and (15b), as well as the same in (15c) and (15d).

Contrary to the generalization in (15), our experimental data show an effect of the number of following syllables even after a single consonant. Across segmental and morphological contexts, schwa is realized more often before one syllable than before two syllables, although the effect is very weak at floor and ceiling, when probabilities are close to 0 (as in C_ at a word boundary) or 1 (as in CC_ at a clitic boundary).

In our model, the fact that schwa is realized more often before one syllable than two follows from stress clash avoidance. The constraint *CLASH favors schwa when it occurs between two stressed syllables.

This approach is similar in spirit to the analysis of Mazzola (1991; 2014), who proposes a stress-based analysis of both schwa and stress assignment, but does not formalize it using constraints.

Stress in French is not fixed at the word level, but falls on the last non-schwa syllable of the phonological phrase (Grammont 1914; Delattre 1939; Jun & Fougeron 2000), with additional stresses on the final full syllable of every lexical word, unless they result in a stress clash (Post 2000). This is shown by the examples in (17), which assume the phonological phrasing and stress assignment rules of Post (2000). According to Post, phonological phrasing is variable, and for a N+Adj sequence, both (17b) and (17c) are possible. We use the term “stress” here for convenience, although some of the references we cite treat prominence in French as pitch-accent (Post 2000) or tone (Jun & Fougeron 2000).

Given that stress always occurs on the last full syllable of the phonological phrase, when schwa is followed by a phrase-final monosyllabic word, it’s also followed by a stressed syllable. As shown in (18), *CLASH prefers schwa before a monosyllabic word in (18a) and (18c), where schwa’s realization avoids a stress clash, but not before a disyllabic word in (18b) and (18d), where stress clash is impossible.

*CLASH has been used in previous analyses of French to account for the realization of both primary and secondary stress (Jun & Fougeron 2000; Post 2000). Primary stress is less likely to be realized when the following syllable is stressed (Jun & Fougeron 2000), and secondary stress is not realized before a stressed syllable, optionally surfacing on an earlier syllable in the phrase (Verluyten 1982; Tranel 1987; Mazzola 1991; 2014; Post 2000). An example of this alternation is shown in (19). Before an unstressed syllable, l’ami is realized with final stress (19a), but before a stressed syllable, stress retracts (19b). When schwa is realized in (19c), stress falls on the final syllable of l’ami, since the realization of schwa prevents a stress clash.

Crucially, the example in (19c) shows that schwa can serve as a buffer between stresses, avoiding a stress clash and making retraction unnecessary.

Côté (2007) presents a number of arguments against a clash-based account like the one outlined above. She points out that the number of syllables following schwa has an effect on its realization, even when there is only one stressed syllable.

Côté (2007) argues that these data cannot be accounted for *CLASH, and instead require a different constraint, such as one enforcing prosodic minimality. As further support for an account with prosodic minimality, Côté (2007) reports pairs like jette de l’ortie and achète d(e) l’ortie, which show that schwa realization is also conditioned by the number of preceding syllables. Schwa is more likely to be realized when it’s preceded by one syllable than when it’s preceded by two. Although Côté’s data seem to require prosodic minimality, such an analysis is not inherently incompatible with our clash-based analysis. Given the seemingly contradictory data in (20) and (21), it is likely that both prosodic minimality and stress clash avoidance are independently necessary to account for the distribution of stress and schwa in French, and a complete analysis of all of the French schwa facts would include both contraints. Since our focus is on the interaction of a few select factors, we leave the fuller account for future work.

2.3 Other restrictions on schwa

One last restriction on schwa, which is relevant to our experimental design, is that schwa generally doesn’t occur next to another vowel, even in contexts where the stress factor favors its realization.

The exception to this generalization is h-aspiré words, which phonetically begin with a vowel (or glottal stop), but pattern in many ways as if they begin with a consonant (see e.g. Boersma 2007 on how h-aspiré words differ from both C-initial words and V-initial words). We set those aside here.

3.1 Experimental design

We conducted the experiment over the internet, using the web-based psycholinguistics experiment platform Ibex (Drummond 2013). The experiment employed a forced choice paradigm, in which participants were asked to imagine that they were speaking with a friend, and choose between two variants of a phrase: one with a pronounced schwa and one without the schwa. Choices were presented in French orthography. Pronounced schwa was indicated with an orthographic e, and unpronounced schwa was indicated with an apostrophe, which is sometimes used to mark deleted schwas in songs to aid rhythmic parsing, or in some colloquially written words (e.g. p’tit for petit). For forms that didn’t contain an e in the orthography, a pronounced schwa was indicated with an e in parentheses (e.g. un toast(e) chaud vs. un toast chaud ‘a warm piece of toast’). During a pre-experiment practice phase, participants received extra instructions for these forms, and were instructed to treat an e in parentheses as a pronounced schwa, and given the example of film(e) russe vs. film russe.⁴

In addition to choosing between schwa and no schwa, participants indicated their confidence in the answer as certainement or probablement. Probablement and certainement responses are pooled in statistical analyses of the data, which model the probability of schwa realization.⁵ A screen capture of the experiment in progress is shown in Figure 1.

Figure 1

Screenshot of experiment in progress.

Previous work has shown that French speakers are capable of estimating the frequency of schwa realization in this manner. For example, Racine (2008) asked speakers to complete a written questionnaire, rating the acceptability of a list of words pronounced with and without schwa. Racine reports that the results of the rating task are very strongly correlated with probabilities of schwa obtained from a production experiment (r = 0.79).

The experiment followed a 2 × 2 × 2 factorial design, with 8 conditions.

The construction of items differed for underlying and epenthetic schwas. Items with epenthetic schwas were constructed according to the template in (24), consisting of a noun followed by a post-nominal adjective, with the site of the epenthetic schwa at the boundary between them.

Depending on the condition, nouns ended in either one or two consonants, and adjectives were one or two syllables long. We controlled for segmental and prosodic context as much as possible. All but two nouns were monosyllabic, and the disyllabic nouns were balanced across conditions. All nouns in the experiment ended in stops, and all adjectives began with obstruents. This ensured that all clusters in the experiment consisted of only obstruents, and in three-consonant clusters, the middle consonant was always a stop, controlling for the influence of sonority on the rate of schwa realization. Examples of the four epenthesis conditions are in (25), with parentheses indicating the alternating e. Each participant saw each noun and adjective only once. The full list of items is included in the appendix.

Deletion items contained the clitic te, the 2^nd person object clitic, which we assume to be underlyingly /tə/. In these items, te was preceded by a name and followed by a verb, e.g. Maurice te cite (‘Maurice cites you’). We used only one type of clitic to control for the fact that clitics may differ in their propensity to undergo deletion (Malécot 1974), and also to ensure that all CCC clusters had similar sonority profiles, with a stop as the medial consonant.

All of the names that occurred before te were disyllabic, and ended in either a consonant or a vowel, depending on the condition. The schwa in te is preceded by one consonant when the name is V-final, and two consonants when the name is C-final. Position of schwa was manipulated by using different tenses of verbs. In the present tense, these verbs are monosyllabic. In the imperfect tense, the suffix -ait /-ɛ/ creates a disyllabic verb. Examples of the four deletion conditions are in (27). Each participant saw every name and verb lexeme only once.

Each participant saw 6 items per condition, 24 for deletion and 24 for epenthesis, in addition to 30 fillers. Fillers consisted of tenses (simple past, simple future) and phonological environments that differed from the test items. Most importantly, some fillers contained phrases with schwa adjacent to vowels, which we used as catch trials. We excluded from analysis any participant who judged that schwa should certainement be realized when adjacent to a vowel. The design is summarized in (28).

3.2 Participants and exclusions

Participants were recruited over the internet through word of mouth. We excluded any participant who did not self-identify as a native speaker of French or chose certainement for the realization of schwa adjacent to vowels in catch trials once or more, leaving data for 27 participants after exclusions. Most participants were either from Île-de-France (7/27), Pays de la Loire (7/27), or Auvergne-Rhône-Alpes (6/27), and participant ages ranged from 21 to 48 (mean = 35.11). Location data is included in the appendix.⁶

3.3 Results

The proportion of schwa responses for both underlying and epenthetic contexts are presented in Table 3 and the barplot in Figure 2.

Figure 2

Proportion of schwa realization from experiment. Whiskers show Wilson score intervals.

Across all four phonological contexts, schwa is judged as better in deletion contexts than in epenthesis contexts. Schwa is also generally judged as better after two consonants than one consonant (the cluster factor), and better before one syllable than two syllables (the stress factor).

To evaluate the statistical significance and effect size of the factors, we fit a mixed effects logistic regression model in R (R Core Team 2017) using the package lme4 (Bates et al. 2015). The dependent variable in the model is the probability of schwa (expressed as log-odds). The model contains the fixed effects in the table in Table 4, each of which corresponds to an experimental condition, in addition to an interaction term for Stress × Seg. The model also contains a maximal random effects structure, with random intercepts for subject and item, and random slopes by subject for all of the fixed effects (including the interaction term).⁷

Given interspeaker variation in the production of schwa, the use of a random intercepts and slopes ensures that the model generalizes across speakers. The inclusion of random intercepts means that speakers with exceptionally high or low baseline rates of schwa will have less of an influence on the estimates of the model predictors, while the inclusion of random slopes means that some speakers can be exceptionally sensitive or insensitive to phonological context. In this way, the random effects structure controls for dialectal and sociolinguistic variation, both of which are well-documented for French schwa. Previous work modeling French schwa has taken the same approach. In Bayles et al. (2016) and Bürki et al. (2011), the inclusion of random effects is shown to help control for interspeaker differences in the realization of French schwa and significantly improve model fit.

The coding of the fixed effects is shown in Table 4. All of the categorical variables in the model were sum coded, as shown in the Coding column. For each variable, the higher level (+1) is the context predicted to favor the realization of schwa.

The fitted values for the model are shown in Table 5. A positive coefficient means the probability of schwa realization increases when the predictor is +1 and decreases when the predictor is –1. A negative coefficient means the probability of schwa realization decreases when the predictor is +1. The rightmost column, Pr > |Z|, shows p-values for the Wald test.

All fixed effects are significant, except the interaction of Stress × Seg. The presence of a preceding cluster has the biggest effect on the realization of schwa; as shown by the coefficient of Seg (β = 1.75), schwa is more likely after clusters than singletons. Schwa is also more likely in deletion contexts than epenthesis contexts (β = 1.48), and more likely when followed by one syllable than when followed by two (β = 0.31). Although the effect of stress is relatively small, it’s significant in the model. The lack of significance for Stress × Seg suggests that the effect of stress is not limited to one segmental context (or vice versa). Both Stress and Seg exhibit independent effects on the probability of schwa realization.

4.1 The models

4.1.1 Constraint set and violation profiles

To illustrate how the models function, we will consider the set of contexts that we analyze as environments for schwa deletion, as opposed to epenthesis. The constraints are given in (29–31). For simplicity, we omit faithfulness constraints here, but include them below when needed.

(29)

*CCC:

Assign one violation for every sequence of three consonants.

(30)

*CLASH:

Assign one violation for every two adjacent stressed syllables.

(31)

NOSCHWA:

Assign one violation for every [œ] in the output.

The contexts are illustrated in Table 6. the schwa is either between stressed syllables (top row) or not (bottom row); and the schwa follows either a singleton (left column) or a cluster (right column).

Table 6

Examples of schwa in the four phonological contexts to be modeled.

Following context	Preceding context
	C_	CC_
_σ́	le vín se vend	la térre se vénd
_σσ́	le vín se vend bíén	la térre se vend bíén

We consider two candidates for each context: faithful realization of an underlying schwa, and deletion. The tableau in (32) shows violations for the two candidates in the context where two constraints are violated by deletion. Violations are marked with negative integers.

(32)	Constraint violations marked with negative integers

The table in (33) uses the more compact representation of difference vectors, which result from subtracting the deletion candidate’s violations from the faithful candidate’s violations. Positive values indicate constraints that prefer schwa’s presence, negative values indicate constraints that prefer schwa’s absence, and zeroes indicate constraints that are indifferent.

(33)	Difference vectors for constraint scores: negative values favor schwa’s absence, positive values favor schwa’s presence

The table of difference vectors in (33) clearly shows the trade-offs in constraint violations in each context. Faithful realization of the schwa always violates NOSCHWA, and deletion always satisfies it, so all contexts have a value of –1 for NOSCHWA, indicating a penalty for schwa presence. This penalty trades off against a reward for schwa realization that depends on the environment. In all of the models we consider, the probability of schwa realization will always be greatest in the environment in which both *CCC and *CLASH are relevant (the topmost row), and will always be the lowest in the environment in which neither is relevant (the bottom row). This sets these constraint-based models apart from a Variable Rules model. As we mentioned in the introduction, such a model could in principle make a schwa deletion rule apply with higher probability in any of the environments (see Coetzee & Pater 2011 for further related discussion). As we will shortly examine in detail, the three probabilistic constraint-based models differ in exactly how rewards can accumulate in terms of differences in probability as we move up the rows.

In Optimality Theory (OT: Prince & Smolensky 2004), schwa realization is optimal iff a schwa-preferring constraint is ranked above NOSCHWA. For example, given the ranking *CCC ≫ NOSCHWA ≫ *CLASH, schwa realization will be optimal in just the top two rows of (33), in which *CCC prefers it, but not the bottom two, in which *CCC is indifferent.

In a deterministic version of Harmonic Grammar (HG; see Smolensky & Legendre 2006; and Pater 2016 and references therein), the optimal candidate is the one whose weighted sum of constraint scores, or Harmony, is the highest. In the tableau in (34), we show the Harmonies that result with a set of constraint weights shown beneath the constraint names. The summed weights of *CCC and *CLASH are greater than that of NOSCHWA, so the candidate that violates them – the deletion candidate in (34a) – receives a greater penalty: –4 vs. –3. Candidate (34b) has higher Harmony (the negative number closer to zero) and is optimal.

(34)	Cumulativity in deterministic Harmonic Grammar

In terms of our difference vectors, schwa realization is optimal when the sum of the difference scores, each times its constraints’ weight, is above zero (see further Pater 2016). For example, with NOSCHWA having a weight of 3, and each of the other constraints having a weight of 2, schwa presence would be optimal in only the corresponding top row of (35), where the result of the just-described equation is +1. In the middle two rows and bottom row, the equation would yield –1 and –3, indicating that deletion is optimal.

(35)	Weighted Harmony differences

This gang effect, or cumulative constraint interaction, cannot be modeled in standard deterministic OT: no ranking of these constraints will produce an output schwa in only the top row; it will always be accompanied by an optimal output schwa in one of the middle rows. In the next section, we will see that a probabilistic version of OT will give the top row higher probability than the middle ones.

4.1.3 Sublinearity through superlinearity in MaxEnt and Noisy HG

We now turn to patterns of cumulativity in Maximum Entropy Grammar (MaxEnt; Goldwater & Johnson 2003) and Noisy HG (Boersma & Pater 2016), two probabilistic variants of HG. In MaxEnt the probability of a candidate is proportional to the exponential of the weighted sum of violations. In terms of the difference vectors, the probability of the realized schwa is eⁿ/(1 + eⁿ), where e is the base of the natural logarithm (approximately, e = 2.71828) and n is the weighted sum of difference scores.⁸ This means that the Harmony difference between two candidates, candidate a minus candidate b, is the log-odds of candidate a. A Harmony difference of 0 produces 0.5 probability, 1 → 0.73, 2 → 0.88, 3 → 0.95, 4 → 0.98, 5 → 0.99, and 6 → 1.0, all rounded to 2 decimal points. Negative Harmony differences equal one minus the positive value (–1 → 0.27, –2 → 0.12, –3 → 0.05 and so on).

So, given the weights (3, 2, 2) used for illustrative purposes in (35), the probability of realized schwa would be 0.73 in the top row (since the difference in harmonies is 1), 0.27 in each of the middle rows (since the difference is –1), and 0.05 in the bottom row (since the difference is –3). This is shown in Table 9. This is superlinear because the cumulative contributions of *CLASH and *CCC (0.46) are greater their independent contributions (0.22).

Table 9

Proportion realized schwa in output distributions with NOSCHWA = 3, *CCC = 2, and *CLASH = 2.

Context	P(schwa)	List of constraint contributions
la terre se vend (CC_σ́)	0.73	Δ*CLASH|*CCC = 0.46
la terre se vend bien (CC_σσ́)	0.27	Δ*CCC|*CLASH = 0.46
le vin se vend (C_σ́)	0.27	Δ*CLASH = 0.22
le vin se vend bien (C_σσ́)	0.05	Δ*CCC = 0.22

Noisy HG is like Stochastic OT, except the values of the constraints are used in a weighted constraint evaluation of the candidate set. Like MaxEnt, it can generate superlinear cumulativity, though as we will see, the patterns the two models predict are not identical.

To begin our comparison of the three models, we first consider the probability distributions they produce when constraints values are set at 1, shown in Table 10. For Noisy HG and Stochastic OT, the noise — the Standard Deviation of the Gaussian — is set to 0.2. For the Noisy HG model, any resulting negative weights were converted to zero (this is called Linear OT in Boersma & Weenink’s 2017 Praat, which we used to explore these models). All probabilities in the table are rounded to two decimal points. The rightmost column shows contributions of the constraints under MaxEnt. This is a case of linear cumulativity, in which the change in probability due to the introduction of a factor (e.g. a preceding cluster) is the same whether or not the other factor (e.g., adjacent stressed syllables) is already present.

Table 10

Proportion realized schwa in output distributions with constraint weights set to 1.

Context	Stochastic OT	Noisy HG	MaxEnt	List of contributions in MaxEnt
la terre se vend	0.67	1	0.73	Δ*CLASH|*CCC = 0.23
la terre se vend bien	0.5	0.5	0.5	Δ*CCC|*CLASH = 0.23
le vin se vend	0.5	0.5	0.5	Δ*CLASH = 0.23
le vin se vend bien	0	0	0.27	Δ*CCC = 0.23

The MaxEnt probabilities arise because realized schwa is preferred by a Harmony score of 1 in the top row, dispreferred relative to deletion by 1 in the bottom row, and the two outcomes have equal Harmony in the middle. Noisy HG also assigns equal probability in the middle rows (as does Stochastic OT). For the top row in Noisy HG, a noise value of 0.2 has a very low probability of subverting the pre-noise preference for the faithful candidate in the top row by making the sum of the weights of *CCC and *CLASH lower than NOSCHWA (less than 0.005, hence rounded to zero). In the final row, no constraint prefers the faithful candidate. In Noisy HG, if a value of zero were sampled for NOSCHWA, the two candidates would be tied, and the tie would be broken with a random choice, which could yield the faithful candidate. The probability of this happening is less than 0.005. To produce a distribution like this in MaxEnt, one could increase the weight values by some constant factor. As the weights get higher, the probability in the top row would approach 1, and the probability in the bottom row would approach 0. Therefore, MaxEnt is capable of representing the more peaked distribution that Noisy HG produces with the current weights, at least to the degree of resolution we are examining.

Because cumulativity in Stochastic OT is predictably sublinear, we know that there is no set of constraint values that will allow it to model the linear cumulativity produced in Noisy HG and MaxEnt with values of 1. It is also the case that Noisy HG is unable to match the distribution produced by Stochastic OT. For Noisy HG, if the weights are small enough to allow NOSCHWA to overcome the cumulative effects of *CCC and *CLASH with 0.67 probability when noise is added, a non-negligible number of faithful schwas will be produced in the bottom row (through random selection in a tie when both candidates have Harmony zero). For example, with the ranking values set to 0.2, the top row gets close to the Stochastic OT value at 0.72, and the middle rows are at 0.50, but the bottom is at 0.16. MaxEnt cannot match this Noisy HG distribution, for reasons we will now discuss.

When the probability in the middle rows is at 0.5, MaxEnt is necessarily strictly linear. This can be understood based on Zuraw & Hayes’ (2017) observation that the contribution of a given weighted constraint violation difference to probability is highest with a baseline at 0.5, and weakens as it approaches 0 and 1. This is due to the logistic function relating Harmony differences between pairs of candidates and probabilities. In Figure 3, we plot probability on the vertical axis, and Harmony difference on the horizontal axis. Note that when we move from –3 to –2 the change in probability (0.07) is smaller thanfrom –2 to –1, (0.13), which itself is smaller than –1 to 0 (0.23).

Figure 3

Probability of a candidate relative to Harmony difference.

The contribution on either side of probability 0.5 is equal: if adding a violation difference increases probability from a baseline of 0.4 to 0.5, it will also increase probability from 0.5 to 0.6. This is the situation we have looked at in the tables thus far, and this explains why MaxEnt cannot match the Stochastic OT (0.67, 0.5, 0.5, 0) distribution in the table, nor the Noisy HG (0.72, 0.5, 0.5, 0.12) distribution discussed in the text.

To escape the clutches of linearity in MaxEnt, we can change the probability of faithful schwa in the non-cumulative context. For example, if we give *CCC and *CLASH a higher value than NOSCHWA, such as 2 vs. 1 in Table 11, the result of adding one of the constraints is a probability higher than 0.5 as in the middle rows, and the effect of adding the other (0.22, the difference with the top row) will be smaller than its effect on its own (0.46, the difference with the bottom row). This is sublinear cumulativity, displayed here by all theories.

Table 11

Proportion realized schwa in output distributions with NOSCHWA set to 1, and *CCC and *CLASH set to 2.

Context	Stochastic OT	Noisy HG	MaxEnt	List of contributions in MaxEnt
la terre se vend	1	1	0.95	Δ*CLASH|*CCC = 0.22
la terre se vend bien	1	1	0.73	Δ*CCC|*CLASH = 0.22
le vin se vend	1	1	0.73	Δ*CLASH = 0.46
le vin se vend bien	0	0	0.27	Δ*CCC = 0.46

MaxEnt can of course match the Stochastic OT and Noisy HG distributions to the degree of resolution we are examining. With the current constraint set, the MaxEnt distribution is completely out of reach of the other frameworks because the faithful schwa gets non-negligible probability in the bottom row, and it is harmonically bounded by deletion. To give them a chance to match it, we can add McCarthy and Prince’s (1995) MAX to the constraint set, which assigns a violation to deletion in every context. To find weights, we used the learning procedure from the next section, with the MaxEnt distribution as the learning target. In a typical run, Noisy HG was able to come close to the MaxEnt distribution with this larger constraint set (0.94, 0.73, 0.73, 0.25), but the Stochastic OT distribution remained fairly distant (0.89, 0.78, 0.78, 0.25), presumably because of its weaker cumulativity.

Finally, Noisy HG and MaxEnt can display superlinear cumulativity in probability differences, as shown in Table 12 in which NOSCHWA is given a higher value than *CCC and *CLASH (again 2 vs. 1). In MaxEnt, we get predictable superlinearity when the result of adding a single constraint is probability less than 0.50. Here, the probability increase from the bottom to the middle rows is 0.15, and the increase from middle to top is 0.23.

Table 12

Proportion realized schwa in output distributions with NOSCHWA set to 2, and *CCC and *CLASH set to 1.

Context	Stochastic OT	Noisy HG	MaxEnt	List of contributions in MaxEnt
la terre se vend	0	0.5	0.5	Δ*CLASH|*CCC = 0.23
la terre se vend bien	0	0	0.27	Δ*CCC|*CLASH = 0.23
le vin se vend	0	0	0.27	Δ*CLASH = 0.15
le vin se vend bien	0	0	0.12	Δ*CCC = 0.15

Since Stochastic OT is predictably sublinear, superlinear patterns are predictably beyond its scope. MaxEnt and Noisy HG can model the Stochastic OT pattern by assigning NOSCHWA sufficient weight relative to the other constraints. With MaxEnt, we can model the Noisy HG pattern by scaling the weights used in the table, which will keep the top row at 0.50, and can bring the other rows as close to 0 as desired, and Noisy HG can in turn model the MaxEnt pattern, at least with the addition of MAX.

In sum, we have shown that each model has restrictions on the types of probabilistic patterns it can model. This means that we should be able to test them in their relative ability to match natural language cumulativity. The biggest difference amongst the models appears to be Stochastic OT’s weaker cumulativity with respect to the other two: it is always sublinear. MaxEnt’s degree of cumulativity, sublinear, linear, or superlinear, was shown to be related to where the effect of a single competing constraint lands in probability space, below 0.50, at 0.50, or above. Noisy HG’s degree of cumulativity is less predictable in that it can model sublinear patterns out of reach of MaxEnt, and in that respect, seems like it falls between the two other theories, as might be expected as it combines Stochastic OT’s noise with MaxEnt’s weighted evaluation.⁹

4.2 Models fitted to French data

Along with cases of underlying schwa discussed in the previous section, our judgment experiment examined four parallel epenthesis contexts, illustrated in Table 13, with the potential schwas underlined.

We assume that the vowels in these cases are not underlying, but are supplied through epenthesis. In the contexts in the rightmost column, the epenthetic schwa avoids a consonant cluster, and in those in the top row, it avoids a stress clash.

The grand means of realized schwa from the experiment are repeated in Table 14, rounded to three decimal points (more precise values were used for finding constraint values). For both underlying and epenthetic schwa, the lowest rate of schwa is in C_σσ́, where the schwa is in the antepenultimate syllable with only a single preceding consonant, and the highest rate is in CC_σ́, where schwa is in penultimate syllable with two preceding consonants. Intermediate values obtain when only the constraint against clusters is relevant (CC_σσ́), or the constraint against singletons (C_ σσ́). The presence of an underlying vowel leads to a higher rate of schwa in all contexts.

The constraint set for these models includes the three markedness constraints introduced in the last section for the deletion cases: NOSCHWA disprefers schwa across the board, and *CCC and *CLASH prefer it in the CC_ and _σ́ respectively. The faithfulness constraint MAX prefers the realized schwa when it is underlying, and DEP prefers its absence when it would need to be supplied through epenthesis (see McCarthy & Prince 1995 on MAX and DEP). We also include *CLUSTER, penalizing a CC or CCC sequence, because schwa is preferred by none of the other constraints in the epenthesis context C_σσ́, so Stochastic OT would be unable to grant it any probability, and would be unable to match the empirical value of 0.090. The preferences of the full constraint set for both underlying and epenthetic schwa are shown in (36). With this constraint set any of the three frameworks can match the data in an individual cell of the table in Table 14 to arbitrary precision, and they can also get the general pattern of cumulative constraint interaction. The question is how closely they can fit the overall pattern.

(36)	Difference vectors for constraint scores: negative values favor schwa deletion, positive differences favor schwa realization

We first present a MaxEnt model whose weights were obtained by using a batch learner (Staubs 2011) that incorporates an optimization algorithm that finds weights that minimize the difference between the training data and the model predictions, in terms of Kullback–Leibler divergence (Kullback & Leibler 1951). This is an implementation of the same general approach to MaxEnt grammar and learning that is presented in Goldwater & Johnson (2003) and Wilson (2006) (as well as Hayes & Wilson 2008, though their model defines a probability distribution over all possible words, rather than over a set of candidates for a given UR). The optimization algorithm was L-BFGS-B (Byrd et al. 1995) as implemented in R. The weights were constrained to be above zero, and a Gaussian prior with variance 100,000 was imposed (the prior seemed to have no effect, as a weaker prior did not change the solution).

Table 15 shows the predicted probabilities for realized schwa in each of the eight environments, as well as the difference between the experimental probabilities and predicted probabilities (positive values indicate that the predicted value is too high, negative too low). The sum of absolute differences for this MaxEnt model with respect to the empirical data is 0.253 (mean over contexts = 0.032; we present SSE and K-L divergence in the summary table at the end of this section).

The constraint weights producing these probabilities are shown in the Table 16. As mentioned in the previous section, the probabilities result from the formula eⁿ/(1 + eⁿ), where n is the weighted sum of difference scores. For the la botte chinoise type of epenthetic schwa (C_σσ́), whose probability is 0.109, the weighted sum is the negative of the weights of NOSCHWA and DEP, plus the weight of *CLUSTER: –1.015 + –1.084 + 0 = –2.099. The corresponding underlying schwa type, le vin se vend (C_σσ́), differs in the absence of the negative contribution of DEP, and presence of the positive contribution of MAX, thus leading to a higher baseline probability for underlying schwa in Table 15.

The contribution of the high weighted *CCC is seen in the probability differences between the columns in Table 15 while contribution of the somewhat lower weighted *CLASH is seen in the probability differences between rows.¹⁰ As discussed in the previous section, the function relating weight differences to probability differences is a sigmoid centered at 0.50 probability. Therefore, the highest possible contribution of a weight difference is when the midpoint between the probability where the constraint doesn’t apply and the probability where it does apply is 0.50. Thus, the greatest contribution of the *CCC constraint is in the penultimate epenthetic context, where it yields a probability increase of 0.608 (0.775–0.167), and the midpoint is closest to 0.50 (0.471). This is in line with the empirical differences, where this context has the highest difference between preceding singleton and cluster. One might think that to get a greater difference between C_σ́ and CC_σ́ for epenthetic schwa than for underlying schwa one would need a separate constraint, but in fact, this follows in the MaxEnt model from the difference in the baseline probability value in each case. Since the baseline probability for underlying case is the singleton probability of 0.633, the MaxEnt model is predicted to yield a smaller probability increase after the cluster. It is worth noting, though, that the MaxEnt model winds up producing a slightly smaller difference between the columns than in the empirical data for epenthetic schwa, and a slightly larger difference for underlying schwa.

*CLASH has its greatest effect on probability differences in the realization of underlying schwa in the C_ environment (0.633–0.514 = 0.119), again because the midpoint is the closest to 0.50. This fits the empirical data in terms of producing a greater effect for *CLASH in the singleton than in the cluster environment for underlying schwa, and also in terms of producing a greater effect for *CLASH in singletons for underlying schwa than epenthetic schwa. One subtle mismatch with the empirical data is that the greatest effect for *CLASH is in fact in the cluster environment for epenthesis (the rightmost column). The MaxEnt model cannot match this because the baseline in that case is further away from 0.50.

To obtain fitted models for Stochastic OT and Noisy HG, we must use on-line learners; no batch approaches are available because it is computationally costly to calculate or estimate model predicted probabilities in those frameworks. In on-line learning, the learner receives a single piece of data at each learning step and uses the grammar to generate a prediction just for that datum, updating the constraint values if the learning datum and the prediction mismatch. Conveniently, it is possible to conduct on-line learning in a nearly identical way across the three frameworks. For MaxEnt, the on-line method is referred to as Stochastic Gradient Ascent (Jäger 2007), and in applying it to Noisy HG, Boersma and Pater (2016) call it the Harmonic Grammar Gradual Learning Algorithm (HG-GLA). The weights are updated by the difference in violation vectors between the learner’s prediction and the learning datum, scaled by a learning rate, or plasticity. In Stochastic OT’s GLA, constraints preferring the correct learning datum are promoted by the plasticity amount, and those preferring the learner’s own incorrect prediction are demoted. When the differences between the candidate vectors are always zero or one, as in our examples (see 34), the HG-GLA and the OT-GLA are identical.

The learning simulations were conducted in Praat (Boersma & Weenink 2017). Constraints were given an initial value of 2, and the plasticity was set to 0.1. The learners received 100,000 samples from the target distributions. These distributions were the experimental results in Section 3, with equal probability given to each off the 8 contexts. The learner then received 3 more sets of 100,000 samples of data, with the plasticity set at 0.01, 0.001 and 0.0001 respectively. This training regime is based on the Praat defaults, but with an initial weight value of 2 rather than 10 so as to get comparable results across the frameworks, and with a correspondingly lower initial plasticity. The noise for Stochastic OT and Noisy HG was set at 0.2, rather than the Praat default of 2, because of the lower initial weight and plasticity. We conducted 20 runs for each model.

The MaxEnt model trained on-line predicts distributions very similar to those of the model trained in a batch fashion. Table 17 shows the results from the model that provides the closest fit to the data, with a sum of absolute differences of 0.240 (mean over contexts = 0.030). The 20 runs had an average summed absolute difference of 0.256 (mean 0.032), with a maximum of 0.269 (mean 0.034).

The weights producing that distribution, shown in Table 18, are somewhat different from those for the batch model, but we again have a relatively high weight for *CCC, and a relatively low weight for *CLASH.

The predictions of the best fitting Stochastic OT model are shown in Table 19. The sum of absolute differences with respect to the empirical data is higher than the best MaxEnt model, 0.299 (mean 0.037). The average sum of absolute differences over 20 runs was 0.330 (mean 0.041), and the maximum was 0.381 (mean 0.048). The distributions of these error measures for the MaxEnt models and the Stochastic OT models are non-overlapping: the worst fitting of the 20 on-line MaxEnt models had less error than the best fitting of the Stochastic OT models. We’ll show shortly that this holds for other ways of measuring error as well.

Like the MaxEnt models, the Stochastic OT predictions get the general pattern of cumulative constraint interactions, and the individual fits are sometimes even somewhat better. The bulk of the error is in the rightmost column epenthetic schwa: the values of the two rows are too close together with respect to the empirical data, which means the effect of *CLASH in the cumulative interaction with *CCC is too weak. In the empirical data, the effect of *CLASH is superlinear: there is a 0.032 difference in the C_ context, and a 0.150 difference in the CC_ context. As discussed in the last section, Stochastic OT produces cumulative interactions that are predictably sublinear in probability space, here leading to a gross mismatch with the empirical data, which show a 0.060 difference in the C_ context, and 0.013 in CC_.

The Stochastic OT constraint values producing this distribution are shown in Table 20. In contrast with the MaxEnt values, the Stochastic OT constraint values are much closer together. This is because variation, and the consequent cumulativity, requires constraints to be relatively close in value so that their ranking will vary across samples from the noise distribution. Nonetheless, we see the same general pattern of *CCC having a higher value than *CLASH.

The final set of predictions is those of the best fitting Noisy HG model, shown in Table 21. The sum of absolute differences with respect to the empirical data is comparable to the best Stochastic OT model, 0.295 (mean error 0.037). The average over 20 runs was also similar, 0.327 (mean error 0.041), as was the maximum, 0.381 (mean error 0.0486). The distribution of error over the eight contexts was somewhat different; the best fitting model is again typical.

The Noisy HG model succeeds in getting a greater spread than Stochastic OT between CC_σ́ and CC_σσ́ for epenthesis. In this respect mimicking MaxEnt, and approaching the empirical spread. In doing this, though, it also creates a greater spread between the values in the C_ column than is motivated by the empirical data. Here Noisy HG is producing a slightly sublinear pattern: the effect of *CLASH on the probability is a 0.088 difference on its own (penultimate column), and 0.081 in conjunction with *CCC (rightmost). In this respect, it is intermediate between the superlinear pattern of MaxEnt, and the highly sublinear pattern of Stochastic OT. Noisy HG patterns like MaxEnt in giving both CC_σ́ and CC_σσ́ for underlying schwa too much probability of realized schwa, and both contexts of C_ too little; these models are not quite fitting the extent to which *CCC has a greater effect in the Epenthetic contexts.

The weights producing the Noisy HG distribution are given in Table 22. As in MaxEnt, the additive nature of constraint interaction in this weighted constraint model allows constraints with even small weights to have an effect on the outcome. Again, the greater effect of *CCC than *CLASH seen in the probability distributions is reflected in the weights, even allowing for the effect of *CLUSTER in singleton contexts.

Our comparisons of models’ fit to the empirical data have thus far been made in terms of differences in raw probability. There are other ways of measuring fit, and one might wonder whether the outcome is different using other metrics. In Table 23, we provide the mean, best and worst fits for each model in terms of sum of squared error and Kullback-Lieber divergence, and also repeat the absolute error values reported in the text.¹¹ In all cases, MaxEnt had consistently lower error than the other models. When error is measured in terms of SSE or K-L Divergence, the Noisy HG values are lower than those for Stochastic OT, and the MaxEnt vs. Stochastic OT difference is enhanced, because the error in the Stochastic OT predictions is concentrated in just two of the contexts (the _CC column for epenthetic schwa).

In sum, all three models – MaxEnt, Noisy HG and Stochastic OT – were able to capture the general pattern of cumulative constraint interaction seen in the empirical data, and provided reasonable fits to the attested values. The MaxEnt model did slightly better than the other models, anxd in comparison to Stochastic OT, at least some of that success is attributable to its ability to produce superlinear cumulativity in probability space.