1. Introduction
Health literacy can be defined as the capacity to obtain, understand and use information to make decisions about one’s health (). A key component of health literacy is functional health literacy which encompasses reading and writing skills necessary for functioning effectively in health contexts. Adequate level of functional health literacy is needed for managing health problems and preventing illnesses, using health care services and making well-informed decisions about one’s health and lifestyle. A key competence among the dimensions of functional health literacy is understanding texts as they relate to health care, disease prevention and health promotion. However, recent surveys around the world have shown that a substantial proportion of the population lacks adequate level of functional health literacy (e.g., ).
Although functional health-literacy in first language (L1) contexts is widely researched (e.g., ), considerably less is known about how people understand health-related texts when they are presented in their second language (L2). In the USA, Sentell and Braun () found that 44.9% of their study participants who reported limited English proficiency also had low health-literacy. In a large number of contexts in Asia, an L2 is used as a mediating language between different language groups in a country (), and consequently health-related information is often exclusively available in an L2. In these contexts there is a large variability in the L2 proficiency of the population () and this can have a significant impact on text comprehension.
To the best of our knowledge, no previous studies have examined the role of general and health-related vocabulary knowledge in the comprehension of health-related texts in an L2 despite the fact that research evidence suggests that limited English proficiency is often associated with inadequate health literacy (). Accessibility guidelines for the presentation of digital information coming into force world-wide (see e.g., Web Content Accessibility Guidelines 2.2 ()) stipulate that texts should be available in a format so that they can be read or listened to, or be processed in both modalities. These accessibility guidelines highlight the potential benefits of bi-modal text presentation to L2 users. However, there is a lack of research on whether presenting health-related texts in dual modality benefits comprehension and whether any potential impact of the mode of presentation varies across readers with differing levels of general and health-specific vocabulary knowledge. This is all the more important because health-related texts are often read without the mediation of clinicians or health-care personnel online. In order to fill these research gaps our study aimed to investigate what role the mode of presentation (reading vs. reading-while-listening), L2 English vocabulary knowledge, and L2 English health-specific vocabulary play in the comprehension of L2 health-related texts in a context where English is widely used as mediating language among different segments of the population.
Another novelty of our study is that it is the first one in this field that was not conducted in a typical Western, educated, industrialized, rich, and democratic (WEIRD) context. As Andringa and Godfroid () demonstrate, there is strong sampling bias in applied linguistics research in favour of participants from WEIRD contexts, which does not only limit the generalizability of research findings but also raises ethical issues. Therefore, our study took place in Sri Lanka where English is used as an important mediating language for social and economic purposes. Sinhala and Tamil are the two official languages in the country and English is a mediating language used between different L1 groups, which is also widely used in commerce, international trade and higher education. Health information is generally provided to the population in Sinhala, Tamil and English. Although most information is available in the L1s of the population, certain types of health-related texts such as medicine labels, prescriptions and hospital reports are presented only or mostly in English. As English is used as a mediating language between Sinhala and Tamil populations, other health-related information texts are often exclusively available in English. Therefore the investigation of how health-related texts are understood and the potential benefits of their multi-modal presentation are of particular relevance in this context. Even though the contribution of general and domain-specific vocabulary knowledge as well as the differential effect of multi-modal presentation has been investigated in previous studies, no research has examined whether these findings are generalizable to a non-WEIRD context and whether the results are applicable to the domain of health-comprehension. Furthermore, our study is the first one that analyzes the role of general and domain-specific vocabulary knowledge in multi-modal comprehension, as previous research has only focused on single mode of text presentation.
2. Background
2.1. Silent reading and reading-while listening
Word-level reading skills play an important role in all key theories of reading comprehension both in L1 and L2. Among L2 readers lack of or gaps in the knowledge of the written form and meaning of lexical items can lead to the breakdown of reading comprehension. L2 readers often have smaller vocabulary size, less rich lexical knowledge and demonstrate slower speed in lexical access than L1 readers (; ), which can result in impaired text comprehension. One way to support L2 comprehension might be the opportunity to listen to a text while reading as it might facilitate the decoding of written words. In L1 research it has been found that hearing the spoken form of lexical items alongside seeing the orthographic form might aid word recognition and speed up the retrieval of word meaning (). This potential benefit of reading-while-listening at word-level decoding might free up attentional resources for higher level reading processes and ultimately support text comprehension in both L1 and L2. One can also hypothesize that reading-while-listening facilitates comprehension based on dual-modality theory (; ). According to dual-modality theory, when information is presented in a dual mode, it is processed through both visual and auditory channels, which then can help L1 and L2 readers retain information and build connections among ideas. Finally, reading-while-listening can offer additional support through non-linguistic cues such as intonation and pausing. As sentence intonation follows phrase-level units, it might help readers to identify chunks of phrases that carry meaning. In a recent eye-tracking study, Conklin et al. () found that L1 participants displayed more and longer fixations and skipped fewer words and made more regressions in the reading-while-listening condition than in the silent reading. This shows that reading is slowed down by the audio input but suggests that L1 users might read more carefully in the dual mode.
The potential impact of multi-modal text input on L1 reading comprehension has been primarily investigated in the field of disability research, where studies have examined whether students with and without disabilities benefit differentially from multi-modal text presentation. Li’s () meta-analysis found that the dual mode of text input had a small sized effect on reading scores of both groups of students, although those with learning difficulties seemed to benefit to a higher degree. The conclusions of Buzick and Stone’s () meta-analysis were similar and showed that in reading-while-listening conditions non-disabled students’ reading scores were .21 standard deviation units higher than in the silent reading condition.
The impact of multi-modal text presentation on comprehension has also been researched in the L2 field. In a study with bilingual Spanish-English students, Reed, Swanson, Petscher, and Vaughn () found no difference in the amount of information retained in reading-while-listening condition compared to the condition where the students read silently. Participants in their study also reported that they preferred silent reading and found listening to a text read out to them distracting. This might have been the case because they were relatively proficient readers and the pace of oral reading was slower than their silent reading speed. Kozan et al.’s () study showed no impact of the dual presentation mode (audio and visual) on the recall of information from a reading text either. Kozan et al. explain their findings with reference to the modality principle according to which “when written verbal information accompanies visual information (i.e., visual-only presentation), they, at least initially, compete for the same resources in the visual channel of working memory (WM), thereby possibly overloading it.” (p. 63). A more recent study conducted in Slovenia by Košak-Babuder et al. () also demonstrated limited benefits of reading-while-listening for young English language learners who did not have an official dyslexia identification. However, with the assistance of bimodal text presentation dyslexic students achieved higher comprehension scores on difficult texts. Nonetheless, there is also some evidence for the beneficial effects of dual text presentation mode for L2 text comprehension. Although Pellicer-Sanchez et al. () found no difference in story comprehension scores in the written and bimodal presentation mode for L2 children, the reading-while listening mode allowed their participants to spend more time processing the visuals accompanying the text. Conklin et al.’s () recent eye-tracking study showed that L2 readers’ gaze was better aligned with the reading text in the dual presentation mode than that of L1 readers. They observed that the eye-movements of L2 readers with lower vocabulary knowledge followed the audio input even more closely than those of readers with larger vocabulary size. This demonstrates that the dual presentation mode might assist lower proficiency L2 learners to stay on task and follow the written text more easily ().
The effectiveness of joint provision of written and oral information has also been examined in a number of studies in health-care settings (for a review see ). These studies were mostly conducted with L1 speakers. One of the main areas of investigation has been how well patients understand information and instructions when they or their children are discharged from hospital. In the majority of these studies, patients receive written information alongside with spoken explanation and health professionals retell the key points in the written discharge letter. Therefore, the procedures are not exactly the same as in educational contexts where the same text is being read out to the students. Nonetheless, a recent meta-analysis of studies on patient discharge information in different modalities, by Hoek et al. () has found that when oral discharge information is complemented by a written text, patients can recall significantly more information.
2.2. The role of vocabulary knowledge in text comprehension
As mentioned earlier, knowledge about the meanings and phonological and orthographic form of words is crucial for efficient text comprehension (cf. ). Lexical quality (LQ) is the degree to which an individual’s knowledge of a given word represents the word’s form, meaning, and the contexts in which the word is used (). Individuals differ in the LQ of the words they know, and readers’ lexicons will include words of varying LQ, from rare words which are infrequently encountered to known, frequent, words (). Quality refers to the extent to which a mental representation of a word specifies its meaning and form in a way that is flexible and precise (). Precision is important in comprehension, because it enables readers to activate the lexical representation corresponding to sensory input, minimizing the chance of activating competing lexical items. Lexical representations also have to be flexible because some words or their definitions are interconnected and may mean the same thing, for example, “flu jab” and “flu vaccine” share the same meaning.
Variation in lexical quality has consequences for text comprehension as words and sentences serve as foundations of meaning (). Mistakes at the word and sentence level may limit processing at the higher level required to build a mental model of the text. In the case of informational texts, such as health-related texts, which also include technical vocabulary, vocabulary knowledge is thought to be a particularly strong contributor to reading comprehension (e.g., ). Vocabulary knowledge plays a crucial role in L2 reading as well (e.g., ; ; ). According to the lexical entrenchment hypothesis, differences between bilinguals and monolinguals can be attributed to the difference in language exposure (). Critically, individuals with less language exposure are likely to have a smaller vocabulary size than those with more language exposure. In turn, those with smaller vocabularies are likely to have lower quality lexical representations and to be less efficient at word recognition and decoding than those with larger vocabularies (; ). A large body of empirical research supports the lexical entrenchment hypothesis and the crucial role of L2 vocabulary knowledge in L2 reading comprehension (e.g., ). Jeon and Yamashita’s () meta-analysis showed that the shared variance between L2 reading and vocabulary measures was approximately 62%.
To the best of our knowledge, no previous studies have examined the role of general and health-related vocabulary knowledge in the comprehension of health-related texts in an L2 despite the fact that research evidence suggests that limited English proficiency can result in inadequate health literacy (). There is also a lack of research on whether presenting health-related texts in dual modality benefits comprehension and whether any potential impact of the mode of presentation varies across readers with differing levels of general and health-specific vocabulary knowledge. In order to fill these research gaps, our study aimed to investigate the following research question:
What role do the mode of presentation (reading vs. read-aloud), L2 English vocabulary knowledge, and L2 English health-specific vocabulary play in the comprehension of L2 health-related texts in a non-WEIRD context?
3. Materials and method
3.1. Participants
In order to represent a range of proficiency levels in our study and make our findings more robust and generalizable, we sampled our participants from two groups of first year undergraduates at two universities in Sri Lanka. One group (n = 160) consisted of students with around A2-B1 level on the Common European Framework of Reference (CEFR) () English language proficiency and the other of students with B2-C1 level English language proficiency (n = 99). Those who had higher English language proficiency came from a private university in which English is the medium of instruction, and at the admission students need to take an English proficiency test. Only students who are at least at pre-intermediate level are offered a place to study. Those who belonged to the lower proficiency group came from a university in which local languages are the media of instruction. The mean age of the low proficiency group was 21.943 (range 20–24) and that of the high proficiency group was 22.162 (range 20–38). In the low proficiency group, 10% of the participants were male and 90% female. In the high proficiency group, 35% of participants were male and 65% female. The students in the low proficiency group were enrolled in the faculty of arts and humanities at a Sri Lankan university and the students in the high proficiency group were from the faculty of management and social sciences at a Sri Lankan university. The proficiency was measured by the respective universities based on in-house tests, which measure the four skills and grammar. The L1 of all participants was Sinhala.
3.2. Instruments
3.2.1. Reading texts
We used four reading texts in this study (see https://osf.io/8nq7x/?view_only=ffdc0c6b98ba499bae9df763ee2a17cf). They were on How to care for your wound (T1); Hypoglycaemia (T2); Pandemic influenza (H1N1) (T3) and What are tonsils and adenoids (T4). All texts were authentic health-related texts. Texts 2 and 4 were chosen from a pool of randomly selected 86 informational texts from the NHS (National Health Sevice) UK websites, and were also used in a previous study by Ratajczak (). These two texts were carefully checked by a health professional in Sri Lanka for relevance to the Sri Lankan context. Texts 1 and 3 were authentic health-related documents written in Sri Lanka and were chosen for the purpose of the research in consultation with a medical expert. Length of the original texts was changed to make them identical with Texts 2 and 4. The texts were judged to be of similar difficulty by participants in a pilot study and had similar readability characteristics as assessed by Flesch-Kincaid Reading grade () and Coh-metrix L2 readability indices (). Care was taken that these two texts would also have similar readability characteristics to Texts 2 and 4.
The texts were of following lengths: (T1) 236 words, (T2) 245 words, (T3) 267 words and (T4) 246 words. Text complexity was checked using the Textevaluator tool (Educational Testing Service, 2013) and ranged between a complexity score of 640–720, representing between Grade 6 and 7 reading level. An analysis with Cobb’s Web Vocabprofile, version 2.5 () also demonstrated the similarity of lexical profile of the four texts: 96.1% words in Text 1, 89.7% of words in Text 2, 89.3% of words in Text 3 and 90.31% of words in Text 4 were from the K1-K3 list, that is, the first to the third thousand most frequently used words in English (results of the analysis can be found: https://osf.io/8nq7x/?view_only=ffdc0c6b98ba499bae9df763ee2a17cf).
Comprehension was assessed by five short-answer questions for each text. This test format was selected as it had been found to yield insights into a wider range of reading processes than multiple-choice items (cf. ). Questions were written in English and answers also had to be given in English. The questions targeted specific information that were judged key to the understanding of the main health-related information conveyed in the text (https://osf.io/8nq7x/?view_only=ffdc0c6b98ba499bae9df763ee2a17cf). The design of the comprehension questions went through several iterations and involved consultations with a medical expert in the UK and a medical professional in Sri-Lanka. Care was taken in wording the questions in a way that they would be easy to understand for the target population. The texts and questions were first piloted with a sample of 20 students from the target population. Revisions to the items were carried out based on the lessons from the pilot that showed that some items were not of the appropriate difficulty and another round of piloting followed with 40 participants. Minor adjustments were made after the second round of piloting for items that did not have appropriate facility values and that contributed negatively to reliability. The Cronbach’s alpha of the comprehension test was 0.793 in the main study. Participants were given a maximum of minutes to answer items for each text.
3.2.2. Recordings of texts
The four texts were recorded in English by Indrarathne whose L1 is Sinhala. As English is widely used as a mediating language in Sri Lanka, a local variety of spoken English was thought to represent higher level of authenticity than a native speaker variety. The recordings were played while the participants read the texts in the reading-while listening mode. The recordings were of the following length: T1: 96 seconds, T2: 110 seconds, T3: 137 seconds and T4: 111 seconds. Reading speed ranged from 116 to 147 words per minute. This approximates the typical narration speed for audiobooks, which is 150 words per minute (Williams, 1998).
3.2.3. Vocabulary Knowledge Test
We used Nation and Beglar’s (2007) Vocabulary Size Test, which is a test of word meaning recognition specifically designed for L2 speakers (validity evidence for the test is provided in Beglar ()). The full test consists of 140 multiple choice items. Each item is a short sentence in which a word is boldfaced. Four options were given under each item and the participants were required to choose the meaning of the word from the four options. We asked participants to complete the first 100 items, which assess the knowledge of words up to the 11th level of 1000 words. Higher levels of the Vocabulary Size test were not used as even C2 level learners on the CEFR () are unlikely to have receptive knowledge of words above this frequency level (cf. ). A maximum of thirty minutes was allocated to this task.
3.2.4. Health literacy vocabulary assessment (HLVA)
The HLVA has been developed by Ratajczak () as a bespoke instrument to measure vocabulary-based health literacy of L1 and L2 English participants (see: https://osf.io/8nq7x/?view_only=ffdc0c6b98ba499bae9df763ee2a17cf). The test was found to be an important predictor of health-related text comprehension for both L1 and L2 readers in Ratajczak’s study, and it discriminated between individuals of different health literacy levels regardless of language background relatively well. The motivation for using a bespoke instrument was to avoid the ceiling effects found in some previous investigations that used standardized measures of health literacy, such as the Short Assessment of Functional Health Literacy (Williams et al., 1999). The HLVA includes 22 lexical items chosen from the Oxford’s Concise Medical Dictionary () that were selected following consultations with a medical expert. These lexical items vary in BNC () and SUBTLEX-UK () corpora frequencies, and during the HLVA administration participants have to define these lexical items. In the current investigation, participants had to define the HLVA items in their L1 Sinhala. The instrument was checked and deemed appropriate for use in the Sri Lankan context by a health professional. Piloting in Sri Lanka prior to the main study showed that the test had appropriate psychometric characteristics.
3.3. Procedure
The study was approved by Lancaster University’s Faculty of Arts and Social Sciences Research Ethics Committee. Participants were asked to give informed consent prior to the beginning of the study. Participants were separated into eight groups to counterbalance the order of the mode of presentation and the presentation of texts (see Table 1). First, the participants were given the participant information sheet and they were asked to sign the consent form. Next, the students filled in a brief background questionnaire about their age, gender et cetera. This took about five minutes. Then, the participants were given instructions to read the texts and answer the comprehension questions. The maximum time allowed for each text was 15 minutes. A five-minute break was given between texts. In the ‘read and listen’ mode, the participants listened to a recording of the text and read the text while listening. The recording was played in the room and the whole group listened to it together. After another 10-minute break, the participants completed the Vocabulary Knowledge Test and the Health-related vocabulary assessment. Each of these tests took 30 minutes. A payment was made to the participants in addition to refreshments.
Table 1
Data collection procedure.
| GROUP | N | PROFICIENCY | CONDITION 1 | CONDITION 2 | CONDITION 3 | CONDITION 4 |
|---|---|---|---|---|---|---|
| 1 | 33 | Low | Read T1 | Read T2 | Read & Listen T3 | Read & Listen T4 |
| 2 | 30 | Low | Read T2 | Read T1 | Read & Listen T3 | Read & Listen T4 |
| 3 | 30 | Low | Read T1 | Read T2 | Read & Listen T4 | Read & Listen T3 |
| 4 | 31 | Low | Read T2 | Read T1 | Read & Listen T4 | Read & Listen T3 |
| 5 | 35 | High | Read T3 | Read T4 | Read & Listen T1 | Read & Listen T2 |
| 6 | 34 | High | Read T4 | Read T3 | Read & Listen T1 | Read & Listen T2 |
| 7 | 33 | High | Read T3 | Read 4 | Read & Listen T2 | Read & Listen T1 |
| 8 | 33 | High | Read T4 | Read T3 | Read & Listen T2 | Read & Listen T1 |
(T1) How to care for your wound (T2) Hypoglycaemia (T3) Pandemic influenza (H1N1) and (T4) What are tonsils and adenoids.
3.4. Data analysis
A trained research assistant whose L1 was Sinhala scored all the tests based on an answer key. The answers to the reading comprehension questions and the responses to the vocabulary knowledge test were scored as correct or incorrect (0 point for incorrect and 1 point for correct answers). The scoring procedure for the health-related vocabulary test has been devised and tested in a previous study by Ratajczak (), using the definitions of the 22 medical terms from the Cambridge English Dictionary (Cambridge University Press, 2018) and Oxford’s Concise Medical Dictionary (). Each item had a maximum score of 2, and to score full marks the participant had to define each item using at least two key components that form each definition. Five percent of the tests were double-scored by Indrarathne and a trained research assistant. Their coding showed some minor differences in the case of four items in the test. These differences were resolved, and acceptable synonyms in Sinhala were added to the coding scheme. The trained research assistant scored the remaining tests using the finalized coding scheme.
To examine the factors that predicted the log odds of reading comprehension accuracy, we used Generalized Linear Mixed-Effects Models (GLMMs). We built these models using the glmer function in the lme4 package () in R (version 4.0.0; ). GLMMs were theoretically appropriate for this analysis because we had item-level accuracy data that followed a binomial distribution. In other words, for each question the only possible outcome was either a correct response or an incorrect response. Thus, we had to model the probability of getting a comprehension question right, and GLMMs allowed us to do that.
4. Results
Our study aimed to answer the overarching research question: How does mode of presentation (reading vs. reading-while-listening), L2 English vocabulary knowledge, and L2 English health vocabulary predict comprehension of English health-related texts? To answer this research question, 259 Sri Lankan students were asked to read, in English, two health-related texts silently, and two health-related texts in a reading-while-listening condition, and then answer five comprehension questions per text (20 in total). This resulted in 5,180 observations. Table 2 shows that our sample of students had higher probability of getting a question right in the reading-while-listening condition compared to when reading silently. Table 2 also shows that the mean differences between reading and reading-while-listening conditions were relatively small. The mean probability of answering a comprehension question correctly of the low proficiency group was .473; 95% CI [.455, .490] whereas for the high proficiency group it was .770; 95% CI [.751, .788]. The mean for Text 1 was .567 95% CI [.540, .594], for Text 2 .619, 95% CI [.592, .645], for Text 3 .553 95% CI [.526, .580] and for Text 4 .606, 95% CI [.579, .632].
Table 2
Descriptive statistics: Mean probabilities across mode.
| MODE | MEAN | STANDARD DEVIATION | ||||
|---|---|---|---|---|---|---|
| Reading | .568 | .496 | ||||
| Reading-while-listening | .605 | .489 | ||||
Note: In each case, the maximum is 1 and the minimum is 0.
The predictor variables in our GLMMs included: mode (reading vs. reading-while-listening), English vocabulary knowledge, and English health vocabulary knowledge (see Table 3 for the descriptive statistics of these two instruments). To minimize the Type I error rate of our predictions, our models considered random variation between participants and questions ().
Table 3
Descriptive statistics for the vocabulary knowledge and health-related vocabulary test.
| TEST | SAMPLE MEAN (SD) | LOW-PROFICIENCY MEAN (SD) | HIGH-PROFICIENCY MEAN (SD) |
|---|---|---|---|
| Vocabulary knowledge (max. 100) | 42.514 (12.081) | 36.244 (6.676) | 52.651 (12.063) |
| Health-related vocabulary knowledge (max. 22) | 7.761 (3.443) | 5.892 (2.234) | 10.783 (2.888) |
In the first step of the analysis, we added a random intercept of participants nested within universities, of questions and of counterbalancing to the random intercept of questions. However, counterbalancing of the eight groups as a random effect led to non-convergence. Therefore, we considered the size of the difference with respect to ordering of the mode only.
Next, we examined whether the addition of predictor variables and interaction terms, while keeping the structural random effects constant, improved the extent to which the observed data matched the values expected by theory, also referred to as the model’s goodness-of-fit. We used the Likelihood Ratio Test (LRT) to compare the goodness-of-fit of the simpler models to the more complex ones (). Specifically, we progressed through a series of models, starting with a minimal model of the log odds comprehension accuracy, with the structural random effects only. The minimal model was compared to an additive model with the effects of the following: mode, vocabulary knowledge, and health vocabulary. The LRT revealed that the additive model provided a better fit to the data than the minimal model (χ2(3) = 71.433, p < .001). Thus, the addition of predictor variables was justified as it improved the goodness-of-fit. Next, we compared the additive model to a model with two-way interactions of mode by health vocabulary, and mode by vocabulary knowledge only. The addition of two-way interactions did not improve the model fit over an additive model (χ2(2) = .700, p = .716). This indicated that the effects of mode were not predicted to meaningfully vary depending on either general vocabulary or health vocabulary knowledge, and that the effects of health vocabulary and general vocabulary knowledge were not predicted to significantly vary depending on mode. Increasing model complexity by adding three-way interactions between mode, vocabulary knowledge, and health vocabulary knowledge, did not improve the goodness of fit over the additive model either (χ2(4) = 1.485, p = .829). Consequently, to avoid overfitting and keep the model parsimonious, we assumed that the effects of mode, English general and health vocabulary knowledge were constant.
In the third step of the analysis, we evaluated whether the inclusion of all structural random effects was justified using the LRT comparisons of models with the same predictors, but with a varying random effects structure. We compared: the additive model with the structural random effects (i); the additive model with random effects of students nested within universities and questions on intercepts (ii); the additive model with random effects of students nested within universities on intercepts (iii); the additive model with random effects of questions on intercepts (iv). The LRT revealed that all structural random effects improved the goodness-of-fit.
Last, as recommended by Barr, Levy, Scheepers, and Tily (), we fit our model with additional random slopes, random differences in the slopes of the predicted effects of general and health vocabulary. We found that the addition of random slopes of vocabulary and health vocabulary knowledge on the random intercept of questions improved the goodness-of-fit of the additive model (χ2(7) = 136.400, p < .001). The attempt to fit other random differences to the model led to non-convergence. Thus, the optimal model contained random slopes of vocabulary knowledge, health vocabulary, and ordering of mode, on random intercept of questions only.
We show the R code used to fit the optimal model below,
Reading Comprehension Accuracy ~ Mode + Vocabulary knowledge + Health vocabulary + (1|University/Participants) + (Vocabulary knowledge + Health vocabulary + Ordering + 1|Questions).
The optimal model accounted for 30.804% of the variance associated with reading comprehension accuracy (calculated using the delta formula; see ). The random effects accounted for the majority of the variance (23.602%), indicating that a lot of variation in individuals’ comprehension accuracy was due to random differences between participants and comprehension questions. The rest of the variance in reading comprehension accuracy (7.202%) was accounted for by the predictor variables, indicating that some variation in reading comprehension accuracy was predicted by the effects of mode, general vocabulary knowledge, and health vocabulary.
We report a summary of the optimal model in Table 4 where we supplement the log odds estimates with odds ratio (OR) estimates and 95% confidence intervals of OR estimates (CIs). We found that although students were on average 1.210 [.893, 1.641] times more likely to answer comprehension question correctly in the reading-while-listening condition compared to reading only, the difference was not statistically significant. In other words, students were on average 4.380% [-2.699, 10.799] more likely to answer comprehension question correctly in the reading-while-listening condition than when reading silently, but the estimate of this effect is highly uncertain and therefore not significant.
Table 4
Summary of the final additive model.
| FIXED EFFECTS | ESTIMATE | OR | 95% OR CI [2.5%, 97.5%] | STANDARD ERROR | Z-VALUE | P |
|---|---|---|---|---|---|---|
| (Intercept) | .492 | 1.636 | [.689, 3.878] | .317 | 1.552 | .121 |
| Mode: Read-aloud | .191 | 1.210 | [.893, 1.641] | .147 | 1.302 | .193 |
| Vocabulary knowledge | .373 | 1.452 | [1.222, 1.737] | .088 | 4.259 | <.001 |
| Health vocabulary | .394 | 1.482 | [1.209, 1.825] | .102 | 3.847 | <.001 |
| RANDOM EFFECTS (INTERCEPTS) | RANDOM SLOPES | VARIANCE | STANDARD DEVIATION | CORRELATIONS | ||
| Participants: University | .299 | .547 | ||||
| University | .106 | .326 | ||||
| Questions | .739 | .859 | ||||
| Vocabulary knowledge | .079 | .281 | –.210 | |||
| Health vocabulary | .103 | .321 | –.480 | .620 | ||
| Ordering of mode | .366 | .605 | –.020 | –.710 | .100 | |
Note: Reading is the reference level for Mode; vocabulary knowledge and health vocabulary variables are centred and standardised. OR refers to odds ratio; CIs refers to confidence intervals.
We also found that with each standard deviation increase in English general vocabulary, students were 1.452 [1.222, 1.737] times more likely to answer a comprehension question correctly (Figure 1). In terms of changes in predicted probabilities, with each standard deviation increase in English general vocabulary, students were predicted to be 8.307% [4.595, 11.910] more likely to answer comprehension question correctly. Similarly with each standard deviation increase in English health vocabulary, students were 1.482 [1.209, 1.825] times more likely to answer a comprehension question correctly (Figure 2). In other words, with each standard deviation increase in English health vocabulary, students were predicted to be 8.740% [4.358, 12.843] more likely to answer comprehension question correctly. These effects were significant, indicating that the effects of English general and health vocabulary knowledge on comprehension of English health-related texts in the general population of Sri Lankan university students, are likely to be positive.
The effects of vocabulary knowledge variation on reading comprehension accuracy across modes, keeping health vocabulary constant.
The effects of health vocabulary variation on reading comprehension accuracy across modes, keeping vocabulary knowledge constant.
Critically, by using the additive model we assumed that the effects of mode are constant. In other words, we assumed that the effects of the mode of text presentation do not vary depending on participants English general or health vocabulary knowledge. We also assumed that the effects of English general and health vocabulary do not vary depending on the mode. Given that our model comparisons revealed lack of evidence for the potential interaction effects of mode with English general and health vocabulary knowledge, evidenced by the lack of any meaningful improvement in model fit, we believe that this assumption is valid for our target sample and population. In other words, we found no evidence to indicate that the strength of English general and health vocabulary effects in predicted probabilities of getting a comprehension question right varied depending on mode, or that the difference between the two modes varied depending on general or health vocabulary of students.
5. Discussion
Our study investigated the question how the mode of presentation, L2 English vocabulary knowledge, and L2 English health vocabulary predict comprehension of English health-related texts amongst Sri-Lankan university students. Before answering this question, it is important to highlight that the mean scores on the health-related comprehension questions indicate that even university students in the investigated Sri Lankan context have difficulties understanding health-related information presented in English accurately. Although the standard deviation figures suggest a relatively large spread in comprehension scores, the overall mean of comprehension performance is 58.629%, which health-literacy standards consider as problematic health literacy (). The low level of understanding of these texts in L2 English is particularly concerning as in another project, Indrarathne and Kormos () found that when the same texts were presented to the participants in their L1, comprehension rate was above 80% even in a non-university educated sample. The inadequate level of comprehension of health-related information can have serious consequences not only for individuals’ health and well-being, but also for disease prevention and control. For example, one of our texts was about what people need to do if they are infected with the H1N1 virus, and in a pandemic situation lack of understanding or misunderstanding of key safety guidance can have a grave impact on the spread of a disease.
One of the possible reasons for the low level of understanding of health-related texts in our sample might be related to the lexical coverage of the texts in relation to students’ estimated vocabulary size. Based on procedures outline in Nation and Beglar (), it is possible to make projections for the vocabulary size of the participants. The vocabulary size of the sample (cf. Table 3) as a whole can be hypothesized to be around 3,000-word families, with the lower proficiency student group having a knowledge of 2,500 and the higher proficiency 3,500 word families. Even though the overall readability indices indicated that the texts were not highly complex, the percentage of the lexical items in the texts likely to be known (i.e., above the 3K word frequency level) by our participants fell between 96.1% and 89.3%. These figures are below the threshold of 95–98% of the word familiarity index assumed to be needed for the adequate comprehension of texts by L2 readers (cf. Adolphs & Schmitt, 2003; ).
Our results reveal that the mean predicted probability of correctly answering comprehension questions was higher in the reading-while-listening condition than in the reading-only condition (by approximately 4.380%), but that this difference was highly uncertain, as the direction of the effect could not have been reliably estimated [-2.699, 10.799]. Although the effect of dual mode of presentation on comprehension in our study is higher than that reported in the meta-analysis of studies in the L1 field (cf. ), it is non-significant. As we found no interaction between L2 vocabulary knowledge and mode of text presentation, which can be considered as a proxy for language proficiency and language exposure (), it seems that the dual mode of presentation of health-related texts does not confer a substantial advantage regardless of the level of L2 competence of Sri Lankan university students. These results are in line with a number of studies in the L2 field (cf. ; ; ; ) and might potentially be explained with reference to Paas and Sweller’s () Cognitive Load Theory that postulates that learners’ cognitive system might be overloaded if they have to process verbal information simultaneously. However, if the effect of dual presentation mode had been detrimental and had interfered with text comprehension, a decrease in scores in the reading-while-listening mode would have been observed. It is more likely that while learners’ attention might have been divided between the listening and reading modes, the two sensory channels of visual and auditory modes might have had a supportive effect (cf. ). These parallel sensory-modality and text representation effects might have balanced each other out, and might have resulted in no substantial differences in text comprehension.
Our findings suggest that when reading health-related texts that require health specific vocabulary, hearing the text read out, does not reliably improve comprehension. Although it is possible that exposure to the phonological form of lexical items facilitates recognizing the form of a word () as also reported by some participants in Vu and Peter’s () study, it does not seem to confer advantages for accessing the meaning of lexical items and ultimately for decoding sentence and text-level information in our investigated L2 context. The fact that the mean scores on both vocabulary measures were relatively low (cf. Table 3) also supports the assumption that if the meaning of lexical items is not known by L2 speakers, phonological facilitation effects do not enhance readers in constructing an adequate mental model of the text. Therefore, simply providing a recorded version of a text in English alongside with the written text is unlikely to raise the comprehension level of health-related information for the general population of university students in the investigated Sri-Lankan context. However, for increasing accessibility of written texts for those with visual impairments and potentially with learning difficulties, it might still be good practice to offer a recorded text of health-related information particularly if the material is available digitally.
In line with previous studies that have demonstrated the important role of vocabulary knowledge in L2 reading (e.g., ; ; ; ; ; ), our research also found that L2 users with higher level of general vocabulary knowledge understand texts significantly better than those with a smaller vocabulary knowledge size. Given that one standard deviation increase in general vocabulary of Sri Lankan students was predicted to increase the probability of answering the comprehension question correctly by approximately 8.31%, the effect of vocabulary knowledge can be considered quite substantial.
Health-related vocabulary knowledge was found to play a similarly important role in the comprehension of health-related texts. The predicted probability of an accurate answer to a question on the comprehension test was predicted to increase by 8.74% for each standard deviation increase in health-related vocabulary. In line with Chin et al.’s () study with L1 readers, this finding demonstrates that familiarity with key health-related terms, which forms an integral part of health-literacy, is necessary for the successful comprehension of L2 health-related texts. The somewhat higher explanatory power of health-related vocabulary knowledge might be due to two reasons. First of all, although health-related vocabulary knowledge and general vocabulary knowledge were found to overlap and share variance in our study, technical vocabulary knowledge also requires background knowledge. Lack of relevant background knowledge may impede meaning integration processes and inference making, and ultimately result in impaired comprehension (e.g., ). In an L2 context it can also act as a barrier for readers/listeners to infer the meaning of unknown lexis. The importance of technical vocabulary knowledge over and above general vocabulary for the L2 comprehension of science related-text was also highlighted by Ardasheva et al.’s () study.
These results can have potentially important implications for the enhancement of health-literacy in contexts similar to Sri-Lanka. First, to promote a higher level of accurate health-related text comprehension in an L2, the L2 vocabulary knowledge of the population might need to be improved. Health-related vocabulary knowledge might also need to be taught together with general vocabulary in English language classes, particularly at secondary school level, so that those who do not go on to study in higher education, would also be able to understand health-related texts to a satisfactory level. Ideally, health-related information is best delivered in the L1 of the readers but this might not always be possible for a variety of reasons. In this case, key medical terms that are needed for the understanding of L2 health-related texts might be glossed and explained in readers’ L1. It is also important to health-education at school to achieve a higher level of functional health literacy across all levels of society in the future.
Our research is not without limitations. Although our sample size was relatively large, the participants were recruited from only two universities in Sri Lanka and not all subject areas that one can study at university were covered. Further research would be needed not only with a more representative university student population but also with the participation of non-university educated population. This would yield better insights into the level of health-related text-comprehension in the general population in Sri Lanka and would enable us to generalize these findings beyond the population of university students. As we mentioned earlier, literacy in L1 is relatively high in the Sri-Lankan context and therefore comprehension of health-related L2 texts might be much poorer in countries where L1 literacy rates and L2 proficiency are lower. In our study, we only measured participants’ comprehension of health-related texts using four texts. A wider selection of texts would allow us to examine the comprehension of different types of health-related information. Finally, more empirical research would be needed that examines differences between the comprehension of health-related information in L1 and L2 and whether explicit teaching of L2 health-related vocabulary or offering L1 vocabulary glosses in L2 health-related texts facilitates comprehension.