1.1. Inferring word meanings from context in a second language

Recently I interviewed fifty Chinese university students studying in an English-speaking country about their attitudes toward guessing word meanings from context when reading in English, their second language (L2). A common concern about this approach to learning new words expressed in these interviews was, ‘What if I guess incorrectly?’ Indeed, researchers have found that L2 readers make incorrect inferences about word meanings during reading (Bensoussan & Laufer, 1984; Frantzen, 2003; Kelly, 1990; Laufer, 1997; Mondria, 2003; Schouten-van Parreren, 1992). Incorrect inferences may arise when the context in which an unfamiliar word occurs provides insufficient support for inferring the meaning (is low-constraining, vague or ambiguous). Another reason for incorrect inferences is text difficulty for an individual reader; Hu and Nation (2000) found that about 98% of the running words in a text need to be known in order to achieve the level of understanding that supports the learning of new L2 words. Therefore, the same text may serve as informative context for one reader, but not another. Incorrect inferences may also arise when an unfamiliar L2 word is mistakenly identified as familiar (Laufer, 1997) or when a guess is made on the basis of form similarity to another word, unrelated in meaning, without adequate attention to context (Frantzen, 2003).

The relationship between contextual inferencing and word learning is not straightforward. Contextual word learning is an incremental process influenced by text, word, learner and situational factors (Paribakht & Wesche, 1999). Correctly inferring a word’s meaning from context is not always synonymous with learning (Frantzen, 2003; Haastrup, 1991; Hulstijn, 2001; Mondria, 2003; Mondria & Wit-de Boer, 1991; Pressley, Levin & McDaniel, 1987). The effect of incorrect inferences on word learning has been researched to a lesser extent, but it is generally assumed that incorrect inferences would interfere with the encoding of correct word meanings and have a negative effect on learning. Studies that looked into the effect of erroneous meaning inferences in reading on word learning tend to support this assumption (Hulstijn, 1992; Carpenter, Sachs, Martin, et al., 2012). Carpenter et al. (2012) provides a detailed insight into the nuanced effects of incorrect inferences vis-à-vis corrective feedback in L2 (German) word learning from reading. Importantly, they found that, once corrected, erroneous inferences were seldom repeated on post-tests. However, the majority of participants whose errors were not corrected (the no-feedback group) repeated their incorrect inferences on post-tests.

One of the limitations of previous studies into the effect of correct/incorrect inferences is exclusive use of off-line, explicit word knowledge measures. Since word learning through reading occurs incrementally, at early stages of learning, an individual encounter with a word (even in an informative context) is often insufficient for the learner to be able to correctly articulate its core meaning, for example, in a meaning generation or translation task. In supportive single-sentence contexts, 3–4 encounters may be needed even for native (L1) speakers to learn word meanings (Bolger, Balass, Landen & Perfetti, 2008; Mestres-Missé, Rodrigues-Fornells & Munte, 2007). When reading in a second language, more encounters may be needed, especially when reading longer texts (e.g., Waring & Takaki, 2003). Nevertheless, readers may acquire some useful information about a new word each time they encounter it in context (L1: Borovsky, Kutas & Elman, 2010; L2: Webb, 2007). Therefore, it is important to go beyond traditional explicit, off-line meaning recognition and retrieval tests in order to evaluate partial word knowledge that may be below the threshold of awareness (Elgort & Warren, 2014; McLaughlin, Osterhout & Kim, 2004; Pellicer-Sánchez, 2015; see also Nation & Webb, 2011). Approaches that use online and offline, explicit and implicit, declarative and procedural word knowledge measures can shed further light on how different aspect of word knowledge are affected by making explicit correct and incorrect inferences during reading.

1.2. The effect of presence or absence of errors during vocabulary learning

Learning new words is underpinned by establishing new memory traces; therefore, learning procedures that avoid errors may result in more robust learning than those that include errors (Baddeley & Wilson, 1994). However, in L1 studies conducted with memory impaired (see Clare & Jones, 2008; Middleton & Schwartz, 2012 for critical reviews) and unimpaired participants, children (Warmington & Hitch, 2014) and adults (Bridger & Mecklinger, 2014; Warmington, Hitch & Gathercole, 2013), results are not clear-cut. Middleton and Schwartz (2012) point out that assumptions of errorless learning (EL) frameworks in cognitive rehabilitation are at odds with testing studies (of non-clinical populations) that show learning benefits associated with retrieval of information from long-term memory and a positive relationship between retrieval difficulty and learning (e.g., Karpicke & Roediger, 2007, 2008). The review (Middleton & Schwartz, 2012) suggests that, in errorless learning studies in amnesia, effectiveness of EL vs. trial-and-error (errorful) learning (EF) depends on the type of patient and training task, i.e., EF is better than EL for patients with mild to moderate memory impairments, while error avoidance may improve learning for patients with severe impairments when implicit learning tasks are used (p. 151).

In studies with normal populations, similar issues arise. Warmington and Hitch (2014) reported some advantages for EL over EF in two deliberate word learning experiments with adult L1 participants. The first experiment involved learning novel word forms (i.e., non-words) for familiar concepts in a word-picture learning paradigm. The EF condition was operationalised by instructing participants to look at an object and provide (guess) its name. Participants were then given the correct object name and asked to repeat it. In the EL condition, participants were provided with the name of each object upfront and asked to repeat it aloud. Word learning was measured using an object naming (production) task and a name-to-object recognition task. Compared to EF, EL resulted in a better performance on the naming task, but not on the name-to-object recognition task. In their second experiment, Warmington and Hitch (2014) used a word-definition learning paradigm and spaced repetitions for learning obscure L1 words (e.g., dossil), requiring the learning of both form and meaning. One familiarisation and two treatment blocks were administered. In the EL condition, participants were presented with definition-word pairs and then asked to retrieve the target word. In the second treatment block, an intervening definition was used to increase the practice interval, “e.g., A coarse dark sugar is called jaggery. A paragraph mark is called pilcrow. What is the name for a coarse dark sugar? (response) What is the name for a paragraph mark? (response)” (p. 589). In EF, the order was reversed; participants were first instructed to retrieve the word from its definition and then given the definition-word pair as feedback. In the second block, the feedback was delayed by including an intervening question-response sequence “e.g., What is the name for a coarse dark sugar? (response) What is the name for a paragraph mark? (response) A coarse dark sugar is called jaggery. A paragraph mark is called pilcrow.” Participants who learned via the EL method performed significantly better on a cued recall test, compared to the EF method. It needs to be pointed out, however, that the design of the two learning treatments in Experiment 2 may have differentially affected word learning outcomes. In the second cycle of EL, a delay in the target word retrieval would have likely been beneficial for learning because it increased the difficulty of the target word retrieval (Karpicke & Roediger, 2007). In the EF treatment, the delay in the provision of feedback could have interfered with learning (Shute, 2008), especially since an additional retrieval event (for a different target word) was introduced prior to the provision of feedback.

Rodriguez-Fornells, Kofidis and Muente (2004) tested word recognition and processing following ER and EF conditions with normal L1 (German) participants learning (memorising) 60 words, using behavioural and electrophysiological (ERP) measures. In the EF condition, participants were first instructed to come up with a number of words beginning with a specified set of letters (e.g., B-R-U), after which they were told which word was the correct answer (e.g., Brust, meaning chest) and asked to repeated it. In the EL condition, participants were given a word beginning with the same set of letters and asked to repeat it. In the recognition phase, participants were exposed to words beginning with the same letters that were either the target word, i.e., the word identified as correct in the learning treatment (e.g., Brust) or a foil, i.e., another high-probability candidate for the same letter-combination onset (e.g., Bruder, meaning brother). A better quality of memory performance (indexed by d-prime) was observed in the EL condition, but an overall number of detected targets was greater in the EF condition. Informed by the finding of more positive ERPs (LPC component) for the correctly recognised target words in the EF (compared to the EL) condition, Rodriguez-Fornells et al., conjectured that the EF condition was associated with a deeper processing of the targets.

Research into error-free L1 word learning with memory impaired and normal participants has mostly been in the form of list or paired-associate learning out of context. Recently, however, Frishkoff, Collins-Thompson, Hodges and Crossley (2016) investigated whether, in contextual L1 word learning involving forced meaning generation (an approach that increases the risk of errors), the effect of erroneous guesses could be mitigated by providing immediate accuracy feedback. They found that the provision of feedback resulted in better word learning (measured by a meaning generation post-test), but only for mixed contexts (with some strong and some weak cues to the meaning of the target word). There was no significant effect of feedback when only high-constraining contexts (with very strong cues) were used, suggesting that words may be learned in such contexts, whether or not accuracy feedback is provided on readers’ explicit meaning inferences (Frishkoff et al., 2016, p. 624). Presumably, because high-constraining sentences are more conducive to the encoding of correct meanings of novel words from context, opportunities to verify inference correctness through accuracy feedback have little to add to learning under such conditions.

Few studies have specifically investigated the effect of errors in L2 vocabulary learning (Boers, Dang & Strong, 2016; Boers, Demecheleer, Coxhead & Webb, 2014; Carpenter et al., 2012; Trenkic & Warmington, 2014). In a conference paper, Trenkic and Warmington (2014) reported an advantage for error-free over trial-and-error word learning, regardless of whether the participants’ meaning guesses were correct or incorrect. In a collocation learning study, Boers et al. (2014) found that making a mistake in completing gap-fill exercises (trial-and-error learning) increased the likelihood of mistakes at post-test, even when corrective feedback was provided and the learners actively corrected their mistakes (cf. Carpenter et al., 2012; Frishkoff et al., 2016). Boers et al. (2016) partially replicated their previous results, but found that learning through trial-and-error was more successful under certain conditions (i.e., when exercises encouraged learners to think about collocations as chunks). Notwithstanding paucity of evidence on the topic, these investigations confirm that the effect of errors in L2 vocabulary learning is not straightforward, and further research is needed.

1.3. Present study

The present study contributes to our understanding of contextual word learning mediated by correct and incorrect meaning inferences, using measures of explicit and implicit knowledge. The key question in the study is how incorrect inferences affect word learning in informative sentence contexts. In the study, Chinese speakers of English read three stand-alone informative English sentences with embedded novel words. Participants were instructed to type inferences about the meanings of these words into a text box (or leave it blank if they could not come up with an answer). After the inferencing task, the participants had a chance to review correct L2 definitions of the novel words; this ensured that all participants had an opportunity to verify their inferences and encode correct word meanings. This design allowed for a tight control over the presentation of the novel words (number and length of exposure; frequency of repetitions; manner in which the task was completed), in order to clarify the effect of inference correctness on learning. After the learning phase, participants’ knowledge of the newly learned words was measured using (1) a meaning generation task that evaluated their ability to retrieve meaning from form (form-meaning mapping) and (2) a primed lexical decision task, in which the repetition-priming effect was used to evaluated their implicit vocabulary knowledge.

It was predicted that the effect of erroneous inferences in this study would not be detrimental for word learning. This prediction was motivated by the following considerations: (1) actively inferring meanings from context (meaning-focused elaboration) increases readers’ engagement with novel words and the context (Hulstijn, Hollander & Greidanus, 1996; Laufer, 2005; Mondria & Wit-de Boer, 1991; Schmitt, 2008); (2) presenting novel words in informative contexts makes learning more likely, due to contextual clues and co-occurrence with known words (L1: Landauer & Dumais, 1997; Mestres-Missé, Rodrigues-Fornells & Munte, 2007; Chaffin, Morris & Seely, 2001; L2: Webb, 2008); (3) contextual word learning is facilitated by the use of definitions as a means of verifying contextual inferences (L1: Bolger et al., 2008; L2: Carpenter et al., 2012; Fraser, 1999; Kelly, 1990; Mondria, 2003).

Furthermore, it is predicted that explicit incorrect inferences are more likely to affect explicit word knowledge (as measured by the meaning generation task), but less likely to predict implicit knowledge that builds incrementally from individual encounters with the novel word in supportive contexts.

2.1. Participants

Study participants (n = 47; female = 36) were either completing an English Proficiency Programme (n = 32) or enrolled in their first university course (n = 15) in New Zealand. Their mean age was 24.5 (St. Dev. = 4 years). All participants were Chinese speakers, whose English language proficiency ranged from intermediate to upper-intermediate, based on their overall IELTS scores of 5.5–7.0. Participates received grocery vouchers for their participation.

2.2. Materials and procedure

All data in the learning and testing phases of the study were collected individually from each participant, in a computer lab.

2.3. Analysis

Linear mixed-effects modelling was used in the data analysis (Bates & Sarkar, 2010; Bates, 2011). All analyses included participants and items as crossed random effects. Random slopes were fitted for fixed effects, as appropriate. A Generalized Linear Mixed Model was fitted to the response accuracy data from the meaning generation task (Jaeger, 2008). The RPT data analysis was conducted after excluding the filler trials and trials with nonword targets. Non-dichotomous variables that were not normally distributed were transformed to bring them closer to normal distribution; they were also centred using the scale function in R to avoid multicollinearity (Belsley, Kuh & Welsch, 1980). A minimally adequate statistical model was fitted to the data, using a stepwise variable selection and the likelihood ratio test for model comparisons (Baayen, Davidson & Bates, 2008). The resulting statistical models contained only variables that reached significance as predictors, improved the model fit or were involved in interactions; all other predictors were excluded from further analysis.¹

In the analysis of the meaning generated task, response accuracy (correct/incorrect) was used as the dependent variable and inference correctness was entered as a primary predictor with three levels (correct, incorrect and no inference entered). The participants’ VLT(2k) score was used in the analysis as a secondary-interest predictor.

In the analysis of the primed lexical decisions, the following questions were examined: (1) whether mixed-modality repetition priming was observed for the newly learned items (but not for the nonwords) and whether inference correctness affected this priming; and (2) whether incorrect inferences negatively affect the accuracy and latency of lexical decisions on the critical items used as targets (across repeated and unrelated trials). The accuracy of responses and response times (RT) were the dependent variables in the corresponding analyses of the RPT data. In both analyses, inference correctness was used as a primary-interest predictor; vocabulary knowledge (the LexTALE score) was used as a secondary-interest predictor in the response accuracy analysis. In addition to being an index of participants’ vocabulary knowledge, LexTALE score acts as a covariate, accounting for the individual variability in their approaches to making word-nonword decisions. The RT analysis also included number of letters (item length), RT on the preceding trial and response accuracy, as additional variables.²

3.1. The meaning generation task

On average, the participants were able to correctly retrieve meanings of about 15% of the critical items in the meaning generation task. There was no statistically significant effect of inference correctness on the first attempt on the accuracy of meaning retrieval. However, there was a reliable effect of inference correctness on the second attempt (Table 1), such that when the meaning of the critical word was inferred incorrectly or no attempt was entered, the resulting knowledge was inferior to that when the meaning was inferred correctly (z = –2.74, p = 0.006 and z = –1.96, p = 0.051, respectively). The meaning generation scores were, on average, about 7% less accurate if the inference entered was incorrect and about 10% less accurate if an inference was not entered (Figure 1A). There was also a reliable effect of existing vocabulary knowledge, such that the participants with higher VLT scores were more accurate in generating the meaning of the critical words (z = 3.62, p = 0.0003).

Finally, there was a significant interaction between inference correctness and vocabulary knowledge (z = 2.16, p = 0.031). For the participants with smaller vocabularies, making a correct meaning inference resulted in a higher probability of retrieving correct meanings at test, compared to when the inference was incorrect or no inference was entered. Interestingly, making an incorrect inference was no worse than making no inference (Figure 1B). However, the negative effect of incorrect inferences diminished with an increase in the L2 vocabulary knowledge, and no difference in the meaning generation score was observed for participants with highest vocabulary scores whether their recorded meaning inferences were correct or incorrect. Conversely, the negative effect of not entering an inference remained even for the most advanced participants (Figure 1B).

3.2. The mixed-modality masked repetition priming task (RPT)

The main finding in the RPT was that repetition priming was not affected by incorrect inferences on either the first or second attempt (Table 2 and 3). A robust priming effect was observed in the response accuracy (z = 4.04, p = 5.3e–05) and latency (t = –2.42, p = 0.022) analyses (Figure 2), and there were no reliable interactions between priming and inference correctness in either analysis. This suggests that contextual word learning progressed whether participants’ explicit meaning inferences were correct or incorrect.³

Meaning inferences on the second attempt had a weak effect on the accuracy of lexical decisions to the critical items (across related and unrelated trials); the responses were somewhat less accurate when no inference was entered compared to the correct inferences (z = –1.70, p = 0.089), although this difference did not reach statistical significance. No response accuracy effect was observed for correct versus incorrect inferences (Table 2, Figure 3A).

In the RT analysis (across related and unrelated trials), there was a weak effect of meaning inference correctness on the first (but not second) attempt, but this effect was not statistically significant. Lexical decisions to the critical items were somewhat slower when the item’s meaning was inferred incorrectly (t = 1.55, p = 0.106) or was not inferred (t = 1.68, p = 0.135), compared to when it was inferred correctly (Table 3, Figure 3B).

4.1. Limitations

This study examined a contextual word-learning scenario that creates favourable learning conditions: the critical items were pre-identified in the sentences and presented in informative contexts, an option to listen to each item was provided during the first contextual encounter, and dictionary-type definitions were presented after the contextual exposure. It is possible that, under less favourable conditions, incorrect inferences may have a larger negative effect on learning. Future research is needed to evaluate the effect of incorrect inferences on contextual word learning outside of the laboratory, under less favourable conditions. Importantly, measures of implicit knowledge should be used alongside more traditional measures of explicit knowledge, in order to charter incremental development of word knowledge from reading. For example, semantic priming can be used to further investigate the development of lexical semantic representations of new words from reading.