3.1. Participants

Data were collected from 36 undergraduates/postgraduates, aged between 18 and 23 (first language Dutch). They were enrolled in a language programme at a Belgian university and all took L2 German courses (linguistics and literature). Eleven students were in their second year of the bachelor’s programme, 10 in their third year of the bachelor’s, and 15 in the master’s programme. Thirteen of the master’s students had studied one semester abroad in a German-speaking country. The students showed mixed language proficiency. Table 1 summarizes the mean scores of a self-rating questionnaire, ranging from 1 (minimal proficiency) to 5 (near-native proficiency).

The mean scores suggest an increasing proficiency from the second year to the final master year. Furthermore, the self-perceived proficiency in the two receptive skills is rated higher than in the two productive skills. These participants did not rate their own speaking skills highly. Judging by the means between 2.9 and 3.5, it shows that the participants rated their own speaking skills in German as average. At the time of data collection, on average, the participants had been learning German as a foreign language for 3.1 (Bachelor 2), 5.0 (Bachelor 3) and 5.6 (Master) years (which includes the time spent learning German prior to the beginning of the respective degree programmes).

3.2. OEIT content

The grammatical content in the present study’s OEIT is known to be one of the most problematic areas for L2 learners of German, namely case marking (see Krumm et al., 2010, pp. 518–736). To identify developmental stages with regard to L2 German case marking, the present study will, analogous to Ellis (2008), build on PT (Pienemann, 1998), because it affords a basis for describing such stages. Recent work in PT has shown a surge of interest in understanding the L2 development of case systems: Apart from L2 German (Baten, 2013), L2 Russian (Artoni & Magnani, 2013) and L2 Serbian (DiBiase et al., 2015) have been studied.

The findings of these studies are quite similar cross-linguistically. Broadly speaking, they all distinguish three stages. In the first stage, learners rely on basic linguistic means (e.g., canonical word order, prepositions) to indicate grammatical functions (i.e., subject, direct object, indirect object). Regarding case marking, three sub-steps exist at this stage: At first, learners only use nominative case markers on all arguments, then direct mapping occur, and finally positional marking. While direct mapping involves a binary case differentiation between initial nominative arguments and non-initial or post-verbal accusative arguments (i.e., there is no differentiation between accusative and dative), positional marking means that cases are linked to the canonical position of the arguments. In other words, with transitive verbs, the first argument is marked in nominative, the second in accusative; with ditransitive verbs, the first argument is again marked in nominative, the second in dative and the third in accusative (as in (1)).

In the next stage, the distinction between accusative and dative case markers emerges in prepositional phrases. This means that learners are then able to associate certain prepositions with accusative case marking (e.g., gegen ‘against’ in (2)) and certain others with dative case marking (e.g., mit ‘with’ in (2)).

In the final stage, learners do not necessarily maintain the canonical word order, which entails that case markers are now needed to indicate the grammatical functions. In this stage, learners show target-like use of case markers in utterances with canonical word order (as in (1)), as well as in utterances with non-canonical word order (as in (3)–(4)).

These three stages (i.e., positional > prepositional > functional) are the basis for the content of the OEIT task that will be described next.

3.3. OEIT procedure

For the present study, a web-based OEIT was administered in a computer lab. The OEIT consisted of 48 stimuli sentences: 16 transitive sentences (e.g., Der Hund verfolgt den Mann, ‘the dog chases the man’), 16 ditransitives (e.g., Die Lehrerin schenkt dem Direktor die Blumen, ‘the teacher gives the headmaster flowers’) and 16 sentences with prepositional phrases (e.g., Der Mann spaziert durch den Tunnel, ‘the man walks through the tunnel’). Half of the sentences were grammatical, the other half ungrammatical with respect to case morphology. Furthermore, half of the transitive and ditransitive sentences comprised a canonical word order, the other half a non-canonical one that topicalized the objects. For each combination of these parameters, there were four stimuli sentences.² Table 2 summarizes the different types of stimulus sentences.

The stimuli sentences consisted of simple syntactic constructions with the most minimal sentence length possible: The transitive sentences contained five words (determiner – noun – verb – determiner – noun); the ditransitive sentences seven words (determiner – noun – verb – determiner – noun – determiner – noun); and the prepositional phrases six words (determiner – noun – verb – preposition – determiner – noun).³ As far as the lexicon is concerned, all nouns and verbs belonged to the basic vocabulary knowledge (Level A1–A2 of the Common European Framework of Reference) according to the German vocabulary list Profile Deutsch (Glaboniat et al., 2005). These items should be known to university learners of German.

The stimuli sentences were recorded by a female native speaker of German in a sound-proof recording studio. They were presented to the participants in a random order. The presentation of each stimulus sentence consisted of three steps. First, the stimulus was presented aurally through headphones (students wore headsets throughout the entire experiment). In the second step, immediately after the oral stimulus, two pictures were shown, visualizing competing semantic interpretations of the stimulus (see Figure 1), one of which was incorrect. The participants were required to choose only one picture. In the third step, while the chosen picture remained on screen, the participants were instructed to repeat the sentence in proper German.⁴

The purpose of the picture-selection task was to achieve a real reproduction/reconstruction, whereby the participants would be delayed in repeating the sentences and draw on their internalized lexicon and grammar while speaking. Additionally, and in contrast to the task design of Erlam (2006) and other OEIT studies, the picture-selection phase was used to measure the comprehension of the stimuli, thereby expanding the utility of the OEIT. The software logged the students’ interpretations of the sentences, allowing researchers to determine whether they selected the correct picture. Their speech was recorded with Audacity. Before the experiment started, the participants were familiarized with the procedure by a trial that included four sentences.

3.4. Analysis

After the experiment, the speech response data were transcribed manually and scored in terms of correct (one point) or incorrect (zero points) case reproductions/reconstructions. Regarding the transcription process, it should be noted the speech data were automatically transcribed. A comparison between the manually and automatically transcribed responses by means of Levenshtein distance calculations revealed a remarkable close fit, thus indicating high reliability of the transcriptions (Cornillie et al., 2017). Concerning the scoring, it should further be noted that sentences that were reconstructed in terms of word order (e.g., when participants produced a canonical sentence after hearing a non-canonical one) were removed from further analysis. The data of the picture-matching task were automatically annotated for either correct or incorrect matches. In the analysis of the speech-response data only the responses that corresponded to correct matches are included, because incorrect matches point to comprehension difficulties. Three analyses are reported on this response data. The first is based on mean correctness scores of the group as a whole. The second used mean correctness scores of the individual participants. The third applied the emergence criterion on the data of the individual participants.

The emergence criterion is the criterion for operationalizing acquisition in PT studies. It can be defined as the “point in time corresponding to the first systematic and productive use of a structure” (Pallotti, 2007, p. 366). However, first use does not actually involve an isolated case but is embedded in a number of different contexts. The present study elicited a minimum of four contexts for each case context. Importantly, because individual cases owe their existence to other cases (Jakobson, 1936), a case cannot be acquired independently, but only in opposition to one or more other cases. In concrete terms, this means that evidence of emergence of a stage can only be assumed when the proportion accusative:dative is 1:1 or higher. This analysis is presented in the form of an implicational scale, which presents the binary result whether or not the developmental stage has emerged (i.e., reached this 1:1 proportion, marked by plus or minus). An implicational scale describes the systematic relationship between stages, such that higher stages imply the presence of lower stages, but not vice versa (Håkansson, 2013a).

Before moving on to the results, it is necessary to address our theoretical and methodological stance toward the analyses used in the present study, because accuracy scores and emergence patterns do not measure the same construct. The emergence criterion describes the beginning of the acquisition process. It shows the cut-off point that remains constant: The underlying processing skills to produce certain linguistic structures are either available or not. Conversely, accuracy scores represent the level of mastery of certain linguistic structures. Naturally, the path from emergence to full mastery is dynamic and characterized by fluctuations. In other words, accuracy scores do not evolve linearly, but are highly variable. Several variables, such as intrinsic feature properties, task properties, learning conditions and individual learner characteristics contribute to the variation in accuracy scores. As such, the accuracy scores of an individual learner on a particular test at a given moment reveal the level of difficulty (Housen & Simoens, 2016). In accordance with this distinction between accuracy and emergence, the present study will consistently differentiate between levels of difficulty when referring to accuracy scores and developmental stages when relating to the emergence criterion.

4.1. Picture-matching task

Table 3 presents the results of the picture-matching task: The first column lists the different types of the OEIT’s stimulus sentences; the two middle columns give the number of correct and incorrect matches; the last column indicates the total number of matches.⁵

The results of the picture-matching task show that the participants have little difficulty in selecting the corresponding picture. In other words, the participants’ receptive knowledge is quite high. However, the results clearly indicate that comprehension is still somewhat difficult when case markers are used purely functionally, namely in non-canonical sentences. Topicalized accusative and topicalized dative arguments yield, respectively, 58 and 53 mismatches. This finding is in line with previous research, which has shown that, in comprehension, both beginning and proficient learners mainly rely on linear word order instead of case information to determine the subject and the object of a sentence (Hopp, 2010; Jackson, 2007; VanPatten & Borst, 2012).

In the present study the mismatches only occurred in the grammatical items of the OVS sentences (i.e., TOP_A and TOP_D, (5)). There is no problem with comprehension in their ungrammatical counterparts (i.e., *TOP_A and *TOP_D, (6)). This disparity of findings can be logically explained by the design of the experiment and more specifically by the competing pictures involved.

In the case of topicalized arguments with correct morphological marking (5), the two pictures visualized competing interpretations, in that both animate arguments can perform the action (i.e., ‘to give the apple’; (a) and (b) in Figure 2). Conversely, in the case of topicalized arguments with incorrect morphological marking (6), the two pictures did not present a competition, because the object in the picture (i.e., the apple in (a) and the flowers in (c) in Figure 2) made the choice quite self-evident. In the experiment, this was necessary because only then would the participants be guided towards reconstructing a non-canonical sentence. More specifically, it aimed for a reconstruction of the morphological incorrect sentence in (6) into the morphological correct sentence in (5). In a scenario with two competing pictures, the sentence in (6) could also be reconstructed into a canonical sentence (der Mann gibt der Frau den Apfel, ‘the-NOM man gives the-DAT woman the-ACC apple’), but this would have deviated from the aim of this sentence type.

4.2. Production data

For the analysis of the production data only the responses to the stimuli of which the picture-matching was correct were considered (k = 1537). Table 4 presents the group mean results for positional, prepositional and functional marking, in both grammatical and ungrammatical contexts (the latter indicated by an asterisk).

The results show that the participants performed highly on the grammatical items, reproducing 97% to 99% of the items correctly, on average. This performance indicates that the OEIT successfully forces learners to reproduce sentences. However, the question is whether the OEIT is also a valid reconstructive test. In this regard, the learners corrected 49% of the ungrammatical topicalized items and more than 80% of the ungrammatical canonical and prepositional items. This ability to correct ungrammatical items suggests that the OEIT is reconstructive. According to Erlam (2006, p. 472) additional evidence of the reconstructive nature of the OEIT is provided when there is a significant positive relationship between participants’ ability to reproduce grammatical items correctly and their ability to correct ungrammatical items. The present study shows a significant positive correlation (r = 0.62, p < .001). Consequently, it can be assumed that the OEIT in the present study is reconstructive and, as such, a valid measure of learner interlanguage.

The present study also examines whether the OEIT can reveal the same developmental stages that were observed in previous research using data of naturally occurring language use. With respect to the three developmental stages of case acquisition in L2 German, a one-way ANOVA reveals that the mean scores of the successful reconstructions are significantly different according to stage (F (2,105) = 21.48, p < .001). Post-hoc comparisons show a significant difference between positional and functional marking (p < .001), as well as between prepositional marking and functional marking (p < .001), but not however, between prepositional and positional marking (p > .05). These findings suggest a distinction between two levels of difficulty. However, mean accuracy scores can be misleading, because individual learners do not necessarily follow the same order of difficulty as the sample as a whole. Therefore, the next analysis compares individual accuracy scores.

Figure 3 shows the individual scores of successful reconstructions in the three types of case marking. The left graph presents the scores for positional marking (from lowest to highest) and the corresponding scores for functional marking. The right graph the scores for prepositional marking (from lowest to highest) and the corresponding scores for functional marking.

In most individuals the scores for functional marking are lower than the scores for positional and prepositional marking. This finding indicates that the distinction of two difficulty levels also exists on the level of the individual learner. However, sometimes the scores are equal and in a few cases the scores for functional marking are higher. Equal scores are not problematic, because this means that those students have full mastery in both stages. Reverse scores may seem more problematic. However, given that these reverse scores only occurred in four out of 72 cases, the impression arises that other variables not controlled for in the current investigation may have caused these results. For example, the first learner in the graph on the left was rather unsuccessful in the picture-matching task, and as a result, only one ungrammatical stimulus for positional marking remained for production. In other words, regarding this learner, there are not enough data to interpret the score for positional marking. Regarding the three learners in the graph on the right, it is unclear what may have caused the reverse scores (although it should be noted that the scores are close to similar in two learners). Nevertheless, the main picture that emerges from these results is that positional and prepositional marking can be considered as a difficulty level that is separate from functional marking.

Zooming in on positional and prepositional marking, the group analysis above did not reveal a distinction between the two. Figure 4 shows the individual scores of successful reconstructions in these two types of case marking, with scores from positional marking presented from lowest to highest.

The scores yielded a complicated pattern: 11 learners scored higher on positional marking than on prepositional marking, 12 learners showed the opposite, and 13 learners showed equal scores. Seeing this pattern, it is not surprising that there was no statistical difference in the group findings. In sum, both accuracy analyses (i.e., the analysis of the group as a whole and that of the individual learner) did not reveal whether there was a distinct level of difficulty between positional marking and prepositional marking. In itself, this result is not surprising, as accuracy scores can fluctuate because of various context- or learner-related factors. It is for this reason that research based on PT makes a distinction between variation and development as two separate dimensions of L2 acquisition (Meisel et al., 1981). In this regard, PT researchers argue that structures that are produced quite accurately at a given moment are not necessarily the structures that were acquired early (and vice versa) (Håkansson, 2013b, p. 118). While accuracy scores are a valid measure to reveal variation in L2 proficiency, PT uses another measure (i.e., the emergence criterion) to operationalize L2 development in terms of developmental stages.

Table 5 presents the abridged implicational scale of the results in terms of emergence (for the full version, see the Appendix). A + means that the case opposition has emerged (i.e., a minimum 1:1 proportion of accusative:dative was reached); - means that it has not emerged. A special case is denoted by (+), indicating that successful reconstructions only occurred in one case. The/means that there were not enough data to determine emergence or not. The bottom row shows the number of participants who represent a certain developmental profile, and the column on the right totals the number of learners who have reached the stage at hand.

The results show that the majority of the learners have acquired all stages (n = 23). This means that these learners show the ability in all contexts to reconstruct sentences with incorrect nominative marking into sentences with correct accusative and dative marking. Six additional learners can be added, but their ability to reconstruct incorrect marking into correct marking does not always show in both cases (i.e., accusative and dative) in all contexts. In the context of positional marking, for example, two learners reconstruct incorrect nominatives into correct datives, but never into accusatives. In the context of functional marking, two learners are able to reconstruct incorrect nominatives into correct accusatives, but never into datives, and vice versa for two other learners. Finally, in the data of one learner there is evidence regarding the emergence of the prepositional and functional marking stage. However, there is no evidence regarding the emergence of the positional marking stage. This is the learner who was unsuccessful in the picture-matching task, and only one item of positional marking remained (see above, Figure 3, left graph). Nevertheless, in all likelihood, this particular learner has also acquired the positional marking stage. Taken together, 30 learners in the present study have acquired the properties of German case marking.

However, in six learners the stage of functional case marking has not emerged. In other words, these learners were not able to reconstruct the incorrect use of the nominative on topicalized objects into the correct use of either accusative or dative. This pattern of 30 learners reaching three stages and six learners reaching two stages suggests an implicational relationship: Learners who have acquired functional marking have also acquired the other two types of case marking. The reverse is clearly not true. An additional implicational relationship between the stages of positional and prepositional marking could not be uncovered, because both stages have emerged among all learners. In other words, the implicational scaling lends partial support to the developmental stages derived from previous PT research. Thus, the OEIT data are comparable to the data of naturally occurring language use that have been collected in PT studies (e.g., Baten, 2013). However, further data collection, particularly with beginning learners, will be needed to provide more robust evidence of the validity of the OEIT.

Finally, it is interesting to relate the results to the participants’ programme level. In Figures 3 and 4 the individual scores of the successful reconstructions were colour-coded: Yellow represents the participants from Bachelor 2, green from Bachelor 3 and blue from the Master year. It can be seen that Bachelor-2 students mainly appear on the left side (i.e., lower scores) and Master students mainly on the right side (i.e., higher scores); in the middle, students from all programme levels can be found. Regarding the developmental stages, the six learners who did not reach the highest stage (i.e., functional case marking) involve four Bachelor-2 students and one student from each of the other two programme levels. While these observations seem to suggest some kind of relationship between level and score or stage, the distribution in the middle and the fact that participants from all levels show the ability to reach the highest stage of case marking indicate that development in terms of individual mean scores and in terms of developmental stages cannot be predicted by the programme level. This is, of course, not a surprise, as levels are based on several requirements within the programme that are not directly related to the language development of specific structures.