2.1. Chinese character learning

Chinese is a logographic language, using characters rather than an alphabetic written system. Learning to read involves form-meaning, form-sound and sound-meaning mapping processes, requiring multi-level representations according to construction-integration models of reading (Chan et al., 2020, Kintsch, 1988). The memory processes in creating such representations are complex, according to the Paired Associate Learning model (Georgiou et al., 2017), which argues that each visual or phonological stimulus needs to be paired with an associated long-term memory response, enabling new items to be stored and retrieved appropriately. Form-sound mappings are particularly hard in Chinese, given its opaque orthography (Perfetti & Zhang, 1995), in that few characters have predictable overt phonological clues in the visual word form. Characters in modern simplified Mandarin Chinese are typically composed of two components: a left-hand semantic component and a right-hand component which may have phonetic characteristics; components may be based, but not exclusively, on core root elements or radicals (Hsiao & Shillcock, 2006). Character properties impacting word knowledge retention and reading ability include frequency of radicals, complexity of strokes, transparency of semantic and phonetic components, and density of connections in semantic networks (Chang et al., 2014; Xu et al., 2014). Establishing strong pairings between these different levels of representation at early stages of reading is therefore highly cognitively demanding (Georgiou et al., 2017).

Existing research on first-language (L1) Chinese character processing confirms that reading is a complex interaction between multiple areas of cognition—visual, motor and verbal/phonological working memory processes (e.g., Perfetti & Zhang, 1995; Siok & Fletcher, 2001). For L2 learners, learning to decode and read in Chinese similarly requires multiple cognitive processes (Bassetti, 2006; Paivio, 1986) as they begin to map visual, auditory and semantic cues. Beginner-level tests of Chinese, such as the Hanyu Shuiping Kaoshi tests at level 1 and 2 (equivalent to Novice on the American Council on the Teaching of Foreign Languages scale, and A1 in the Common European Framework of Reference for Languages) acknowledge this challenge and do not require character knowledge until higher levels of proficiency. Studies show that strong links remain between visual processing of a character/word and knowing its pronunciation or meaning, even at intermediate to advanced levels (Everson & Ke, 1997; Zhang et al., 2017). However, there does not yet seem to be extensive research on how visual and linguistic processing may combine at initial learner stages, despite the struggles beginner-level learners face in mastering Chinese characters (Wright et al., 2022).

One route to investigate how characters are learned relates to whether characters are processed in parts (analytically) or holistically. Research indicates that a higher number of strokes, particularly for less frequent words, can require slower analytic processing (Jiang & Feng, 2022) even among intermediate-level learners. Su and Samuels (2010) suggest that “beginning Chinese readers process characters in an analytic way, but that the decoding process changes gradually from analytic to holistic as their reading skills develop” (p. 1085).

To improve their decoding and reading skills, L2 Chinese learners adopt many useful strategies, including visualisation, when learning or being tested on words/characters (e.g., Shen, 2005; Shen & Ke, 2007). Shen (2005, p. 56) found the most common strategies included “paying attention to graphic structures”, “visualizing the graphic structure of the character” and “making use of the phonetic and semantic information in radicals”. Visual and semantic information, therefore, are tightly bound in learning characters (as the paired associative learning model would predict). However, the link between character composition and word meaning is complex, where one character with one meaning can take on a different meaning if combined with another, leading to plenty of ambiguity in character interpretation and making the semantic-form connections difficult to establish (Georgiou et al., 2017). Character instruction often uses written repetition, following the traditional approach to practising fixed stroke order to fix character knowledge (Ke, 1998). The risk here is that characters and words are often presented as decontextualised lists, leading to a risk of rote memorisation in which the character form may not be well connected to meaning (Kuo & Hooper, 2004; Shen, 2004). Research shows that learners struggle to surmount the perceived character learning challenges well beyond initial stages (Wright et al., 2022). Teachers could therefore motivate their learners in early-stage learning by focusing on some of the most frequent, easily identifiable and semantically predictable radicals within a character’s composition, easing the transition to character recognition and learnability (Shen & Ke, 2007; Zhang et al., 2016).

2.2. Visual processing using eye-tracking methodologies and VWM

To understand better the visual/cognitive processes involved in “paying attention to graphic structures” (Shen, 2005, p. 56) and how such processes are involved in learning character form and meanings, we argue that eye-tracking methodologies (Godfroid, 2019) can provide fresh insights into these processes in real time. Recent studies of eyetracking used by learners generally find that longer gaze patterns can indicate more focused attention, indicated by slow or repeated text reading (e.g., Bax, 2013; Stickler & Shi, 2014). We suggest that longer eye gaze would therefore indicate visual attention to specific character components in the learning process, particularly among novice readers using the kind of analytic learning discussed earlier. However, to our knowledge, eyetracking methodologies have not been used to investigate word learning processes in beginner-level learners of Mandarin.

We also argue that focused visual attention using longer eyegaze would entail greater use of working memory, particularly VWM. Working memory (WM) has been argued to play a potentially critical role in L2 learning and processing where attention required (Skehan, 1998; Wright, 2015). Attention is controlled by a central executive system, binding phonological and/or visual information from short and long-term memory to complete a task (e.g., learning words). Visual shorter-term memory (STM) storage is assumed to be more space-constrained than phonological (around three chunks for visual STM compared to around seven for phonological STM, Zhang & Simon, 1985). Therefore, visual storage capacity alone may be relatively hard to use in linguistic studies to predict individual variation in terms of character recognition or learning. Nevertheless, we believe it is logical to assume that greater WM executive capacity (combining storage with processing) and VWM specifically may be associated with greater ease in character reading and learning processes. And if so, it may follow that VWM is associated with specific eyetracking behaviour (e.g., greater VWM capacity may facilitate shorter and fewer fixations to free up WM capacity elsewhere).

Some studies using WM in relation to Mandarin support the logical assumption that greater WM capacity supports ease of character learning, as with most other forms of learning (Nelson & Shiffrin, 2013). Reder et al. (2016), using a Paired Associate Learning paradigm, argued that familiarity with components aids learning of novel words made up of multiple components (e.g., bi-morphemes) by reducing WM load in processing the novel input. Their design was a lab-based experiment using university students with no prior knowledge of Mandarin, trained in identifying visually similar or different characters in a visual search task, for an hour a week for four weeks. Participants’ ability to create form-meaning pairings for novel bi-morphemic words was tested in a recall task for words including familiar frequently-presented characters or unfamiliar characters. They found that pairs combining more familiar characters were more easily learned. Performance in the recall task also associated with performance on a WM n-back task requiring recognition of Chinese characters in a sequence, indicating that familiarity or ease or recognition reduces WM pressure in the learning process. This study supports the logical connection made here that visual processing and WM capacity are connected in learning characters. However, the controlled experimental conditions make it difficult to extend the value of their experimental findings to more typical vocabulary learning processes for early-stage learners.

In another study of WM effects, including VWM, Kim et al. (2015) investigated WM in character processing among intermediate-level learners (with prior knowledge of 1500 characters), but without eye-tracking. Testing involved VWM and verbal WM, associated with speed in recognising input-enhanced characters (where one stroke was made bold), phonologically connected characters or semantically connected characters. VWM was found to have an effect on recognition of the enhanced characters but not on semantic or phonological connections. This would suggest that that VWM may likely be associated with intentional focus, but perhaps more in relation to visual signals, rather than in the kind of semantic pairing needed in word learning.

While many existing visual cognition studies used participants who already had a relatively workable Chinese vocabulary, we wanted to see if a similar rationale could explain learning processes required at beginner-level. Specifically, we aimed to explore if visual character processing would be easier when including consistent semantic information in radicals, which could help beginner-level learners to “notice” and recognise character-meaning connections more easily, or whether visual salience alone would draw learners’ visual attention to help learn new words.

2.3. Individual differences and role of exposure

Alongside internal cognitive factors that may impact word learning, individual variation in external factors, such as amount of exposure in and outside the classroom, may also play a role (Briggs, 2015; Wright, 2013). Research into initial language learning in Chinese shows that even very little exposure (fewer than 10 encounters with a word) can lead to sound-meaning word knowledge (Han & Liu, 2013), though this did not include written character learning. As noted earlier in Reder et al. (2016), characters can be decoded accurately by novice learners with minimal training, albeit in laboratory training setting. Immersion, such as in study abroad settings, has generally been found to boost lexical acquisition, even in short stays of less than eight weeks, though findings can be variable and may depend on quality and quantity of exposure during immersion and whether learners are true novices (Briggs, 2015).

Thus, there appears to be several gaps in understanding the underlying cognitive processes supporting making initial paired-associations between form and meaning at the earliest stages of learning Chinese and in how focused visual attention and VWM may be implicated, particularly in a fully-immersed Chinese setting. Based on our interpretation of the literature discussed above, we suggest that careful focused attention in visual processing as measured in eyegaze patterns and VWM may be connected to the kind of visual strategies reportedly used by learners when processing characters, assuming that beginner-level learners use visual salience as a way to bootstrap meaning, but this is an issue that needs further exploration.

Therefore, to address this gap, this study investigated learnability of three specific target radicals in monomorphemic words, comparing visually similar but semantically different radicals for animal (nominal) and hand (verbal) and a visually salient but less semantically predictable radical for mouth (verbal). Extending previous research by Kim et al. (2015), Reder et al. (2016) and Zhang et al. (2016), our study is the first, to our knowledge, to test visual versus salience semantic predictability in character learning, using eyetracking patterns as evidence of focused attention, in combination with visual working memory processes, among beginner-level learners in an immersion setting.

Our research questions and associated predictions are:

• (1) How do beginner-level learners recall different types of monomorphemic characters, measured in accuracy and reaction time of responses? Is there an effect of type (semantic predictability or visual salience)?

We predicted that overall salience would impact on recall more than semantic predictability, but there may also be evidence of a general noun-bias if semantic predictability plays a role.

• (2) Do eyegaze patterns indicate ease of recall?

We predicted that better recall would pattern with reduced focal attention (i.e., fewer fixations and shorter durations on the target area, in proportion to overall eyegaze).

• (3) Are there effects for VWM on recall or eyegaze?

We predicted that greater VWM capacity could facilitate either higher character recall scores, or reduced eyegaze, or both.

3.1. Character learning test

We created a self-paced learning task using PsychoPy to test speed and accuracy in recalling 30 target monomorphemic characters, along with 10 distractor characters. The test characters were divided between three types of radicals, always found in the left-side component of the character.

Character stimuli consisted of:

• 10 verbs using 扌hand radical (TargetV)
• 5 nouns using 犭animal radical (TargetN), visually similar to TargetV but a different functional/semantic category
• 5 verbs using 口mouth radical (TargetM), in the same functional/semantic category as TargetV, with high visual salience, but less consistent semantic meaning

The characters were a mix of more or less frequent words, but most would not yet have been taught in textbooks (indicated by personal communication from the head of the participating institute, in which a standard textbook was used for all beginner-level classes). The hand and animal radicals were chosen as reasonably visually similar, though the hand radical is usually rated as more frequent in Chinese dictionaries (Modern Chinese Frequency Dictionary, 1985). The mouth radical was chosen as very visually salient. It is also found in a character included in early taught input for making introductions (叫 “jiao”, shout or call – as in “wo jiao Ma-ke” – I am called Mark) and was used as a baseline to test if learners had started to achieve some character familiarity. Table 1 shows all the characters used in the study.

Two bilingual Mandarin/English speakers confirmed that the selection of characters showed a reasonable mix of visual salience, semantic consistency, and frequency. TargetV (verbs, hand-radical) words chosen here are more frequent, but of medium semantic consistency in connection to verbs linked with the idea of “hand”. TargetN (noun, animal-radical) words are less frequent but more semantically consistent in connection to types of animal. The radicals in these two types of words were judged by the two raters to be visually similar in stroke form and salience, but the semantic types were distinct. This would allow us to test our prediction of a potential general noun-preference bias in early-stage word learning, which may be evident in the comparison of scores for the two types of words.

TargetM (verb, mouth-radical) morphemes, by contrast, were chosen to represent frequent, very visually salient words but with lower semantic consistency in connection to verbs linked with the idea of “mouth”. The 10 hand-radical verbs were taken as the baseline for learning. The 5 animal-radical nouns aimed to identify differences by semantic type, while the 5 mouth-radical verbs aimed to identify differences by visual salience.

The test was carried out using a standard Windows-based PC, connected to an Eyelink 2000 tower-mounted system, with a sampling rate of 1 kHz (SR-Research, Ontario, Canada), screen ratio 1920 × 1080. Although viewing occurred with both eyes, eye movements were recorded from the left eye only. Items were presented on a 21-inch LCD monitor, positioned 71 cm from participants, subtending approximately 1° of visual angle. Characters were presented in SimSun 350 font, using black on grey ground, after piloting various sizes and colours to ensure legibility and accuracy of target gaze fixation (Godfroid & Hui, 2020). This format was judged to be clearly legible and large enough to distinguish eyegaze direction between the left and right-side components.

In the learning phase, an image of the Chinese character was presented on screen first, then the English word; participants pressed the space bar to advance to the next screen. We then tested recall of those target words, presenting the English word, then a Chinese character, asking participants to judge if the character matched the meaning, by pressing a Shift key (Right for match, Left for unmatched). The recall test was carried out after an intervening 15-minute gap used to conduct a VWM test (see below) and to gather biodata and other relevant participant information. Both the teaching phase and recall testing phase used randomised presentations of the target words to avoid practice effects.

Participants were given time to get comfortably settled at the computer using the Eyelink headrest; they were given three practice character/word pairs first to feel at ease using the computer and keyboard buttons. They were given three sample tests to practice ahead of the real test because it required some practice to keep hands still on the keys without looking down, which would alter the pace of the test. They were instructed to complete the test at a brisk but natural pace. We recommended that they do the untimed practice test in about two to three seconds per item, which many managed to do. The screen was automatically set to move to the next item after six seconds. Results from the recall test were automatically analysed by the PsychoPy test software to create accuracy scores and reaction times in seconds, allowing us to test our prediction that greater salience would lead to higher accuracy and faster reaction-time (RT) scores on the TargetM radical words compared to others.

3.2. Eyegaze procedure

For the eyegaze analysis, we set specific Areas of Interest (AOIs), encompassing the key radical on the left-side component, as shown in the example in Figure 1 for 叫 (“jiao”, shout, be called).

We collected data on first fixation, length of first fixation, number of total fixations and total gaze time per item to give insights into proportion of time spent on target AOI during recall. The outcomes allowed us to test our prediction that better recall scores (higher accuracy, lower RTs) on the character recall test would pattern with reduced focal attention (i.e., fewer fixations and shorter durations on the target area, in proportion to overall eyegaze).

3.3. VWM procedure

To test VWM, we used a visual shape search and recall task (adapted from Gorbunova et al., 2019), tapping individuals’ capacity for retaining memory for a specific version of a particular shape while doing a distractor letter-spotting activity. This was selected to mirror the kind of visual shape knowledge entailed in distinguishing characters’ stroke shape and position. The VWM recall test consisted of an automatically-timed task in which one of four types of shape were presented in different positions on the screen (diamond, square, rhombus or star), with four variants per shape (e.g., vertical or horizontal orientation, skewed left or skewed right). Participants then did a brief distractor test, looking at an image of circles on the screen with one circle labelled M or Z (randomly presented in different positions around the circle) and pressing the matching keyboard letter. Finally, they viewed a set of four variant shapes presented horizontally across the screen and selected which one had been previously presented (using Z, X, N, M letters to indicate diamond, square, rhombus or star respectively). The target recall shape was presented for 250 milliseconds, then the circle screen automatically appeared for four seconds, and finally the target choice screen appeared for four seconds. Participants could also press the space bar to proceed through the test at their own speed. They were given three sample tests to practice ahead of the real test, as before, to feel comfortable keeping their hands on the keys without looking down. Accuracy and reaction times in pressing the correct key at the final target choice stage, recalling the previously presented shape, were automatically calculated by the test software.

3.4. Participants

Bio-data were gathered using a simple questionnaire to identify suitability for the study (adapted from Wright, 2013). Recruited participants were Anglophone speakers, L1 English or dominant L2 English bilinguals with alphabetic-based L1s. Age was not considered to be a relevant variable because all had to be over 18 to attend university classes. Almost all had arrived in China one to two weeks just before starting classes. All participants had 15 hours per week of in-class instruction. They were tested within five to seven weeks of starting their beginner-level Chinese programme, to maximise homogeneity in terms of learning status. Despite our best efforts to recruit novices, there were some participants with some knowledge of Chinese (e.g., having travelled in China in previous years or having studied Chinese in their home country several years ago). However, they reported that they had not learned written Chinese or had not kept up Chinese over the years, hence they were all assigned to beginner-level classes. To mark this variability in prior knowledge, a between-group variable of novice/non-novice was tabulated, where more than two month’s knowledge of Chinese counted as non-novice (i.e., longer than the in-class sessions all our cohort had received at the time of testing). Length of months’ residence in China and daily levels of exposure (on a scale of one, for less than five hours a day, to three, for more than 10 hours a day, following Wright, 2013) was also recorded but proved to have no relation to later results.

Full ethical and consent procedures were followed according to university protocols; participation was voluntary with an inexpensive coffee-shop voucher offered as a thanks. No class teachers were involved in conducting the experiment, and all participants were informed there was no connection between the experiment and class progression.

Forty learners participated, originally 21 novices (no prior knowledge) and 19 with some prior knowledge. Missing data in the eyegaze recordings and VWM tests and some outlier extremely slow responses led to five participants being removed; remaining scores presented here are from 17 novices and 18 non-novices, 35 in total (10 male, 15 female). Data were found to be non-normally distributed, likely due to the small number of participants, and were analysed in SPSS using non-parametric measures of association or difference.