2.1. Pseudoscience: Delimitation problem

The delimitation of science and pseudoscience is the topic of book-length publications (cf. Pigliucci & Boudry, 2013, for a recent example). Applied linguistics and the study of L2 acquisition are sometimes referred to as soft science – they may not be (hard) science in the narrow sense of the term, but they should also not be pseudoscience (Mahner, 2013, p. 25). Occasionally, scholars in our field take a Popperian stance (Hulstijn, 2015, p. 26), which relies on one main criterion to demarcate true scholarship from pseudoscience: scientific theories should be falsifiable. Others apply less strict criteria (e.g., converging evidence from multiple studies as a basis for the confirmation of theories (Mahner, p. 26)). Research on L1, L2, and L3 acquisition deals with phenomena that depend on a multitude of factors that are often difficult to measure. For research in our field to yield results that can inform educational policies, I suggest the following three assumptions as minimal requirements: (1) central tenets of the main theories in our discipline are amenable to measurement, (2) the theories that deserve attention yield hypotheses that can be tested, and (3) it is possible to make generalizable claims about language learning that allow prognosis and not just retrodiction.

I am aware that these three points are not uncontroversial. There are several approaches to L2 and L3 acquisition that I deem to be scientific but that do not fulfil these three criteria, which is why they lie outside the scope of the present contribution. As an example, qualitative approaches that seek to shed light on the mechanisms of interactions in selected tokens of discourse (Pekarek Doehler, 2010) certainly question requirements (1) and (2) and therefore probably also requirement (3). Another example would be dynamic systems theory (DST) based approaches. They are often characterized as meta-theoretical frameworks that “unify” (de Bot et al., 2013, p. 216) mid-level theories and therefore do not directly yield testable hypotheses. Moreover, some scholars working within DST approaches focus on retrodictions rather than predictions (Larsen-Freeman, 2016, p. 388), which is due to the complex and hard to predict nature of the language-acquisition process in this view. While I have great respect for in-depth qualitative analyses and in the investigation of individual and intra-individual variability, I believe that it is also worth trying to make generalizable claims about language learning. Such generalizable claims, however, require evidence from high-quality quantitative research. At the risk of repeating myself, I do not claim that qualitative research or case studies are unscientific, but that policy recommendations need to be based on a different type of scientific evidence (requirements 1–3 above). All that follows here is based on this assumption.

2.2. Definitions

I suggest three rough working definitions categorizing research and research-like activities in our field. Science involves constructing theories explaining puzzling phenomena and empirically testing hypotheses derived from these theories. Scientific inquiry does not have to start from theory, it can also involve bottom-up observation and exploration of phenomena, which later allows for theory construction and revision. Scientific knowledge is unstable because it often advances by proving the old theories wrong and proposing new, better theories. Scientists change their beliefs if there is enough evidence to do so. What enough evidence means is controversial, especially in humanities. There is good science and bad science, but both are science and are amenable to critical assessment by the scientific community. However, if a theory is compatible with all potential data patterns in empirical investigation, it belongs to the realm of pseudoscience. Pseudoscience is based on theories that are vague, that yield untestable predictions or predictions that always apply. The output looks like science, but it does not contribute to the advancement for the field since theories, beliefs and assumptions cannot be proven wrong. Both bad science and pseudoscience need to be distinguished from what Frankfurt (2005) calls bullshit. Bullshit refers to texts produced by people who want to impress without caring about facts, whereas authors of bad science and pseudoscience indeed do care about facts and truth. Indeed, in the case of pseudoscience they often care too much (see below).

2.3. What does “Research has shown…” in policy discourse refer to?

If a policy recommendation bears the badge “research has shown…”, the research in question might simply not exist, the recommendation might be based on misinterpretation of solid research or on pseudoscientific beliefs, on bad science, but it is also possible that there are indeed robust findings underpinning the policy recommendation.

A policy recommendation in the domain of language teaching and learning that claims to be evidence-based typically involves the following elements:

• A curriculum-related measure Z (e.g., teach HLs to children of immigrants, start teaching two FLs on the primary school level)
• Reference to research that has shown a causal effect of one entity on another (X→Y in Figure 1; e.g., early age of onset → better proficiency; L1 → positive transfer to L2; first FL → positive transfer to second FL)
• The assumption that Z allows unfolding the effect of X on Y (i.e., Z→(X→Y))

The terms programmatic literature and doxa in Figure 1 might need some explanation. As will be shown in two examples in the next section, there are cases in which the efficacy of a measure Z only relies on literature that is programmatic (e.g., literature that makes claims on Z, possibly based on evidence on X→Y, without providing any evidence on the effects of Z). If groups of actors in the scholarly field take such programmatic claims and their entailments for granted, then this shared common ground represents what Bourdieu (1979, p. 549) refers to as the doxa.

As Figure 1 shows, there are many cases where no recommendation should be made. Three configurations, however, license recommendations with increasing certainty. In Figure 1 I do not address the possibility that a policy measure Z has beneficial effects even though the evidence as to X→Z is inconclusive. Such cases undoubtedly exist, just as certain substances have proven to have a therapeutic effect although research does not fully understand why. The focus of my contribution, however, lies in the possibility for second language acquisition (SLA) researchers to make evidence-based recommendations, for which having converging evidence of X→Y is a prerequisite.

3.1. Lack of fit: Surrogate outcomes

When it comes to the effects of L1s on L2s (X→Y), there are some robust findings. There is ample evidence showing all kinds of cross-linguistic (bi- and multi-directional) influence on many different levels, from phonology to syntax and pragmatics (e.g., Cenoz, Hufeisen, & Jessner, 2001). But this evidence mostly pertains to fundamental research, not to teaching-oriented research. Moreover, this research includes evidence about what is often termed negative transfer. Evidence from cross-linguistic influence in SLA and bilingualism research is not a fit basis for the policy recommendations. It is almost never based on intervention studies, but on studies of bi- and multilingual subjects, and these studies often involve rather controlled tasks: Psycholinguistic experiments or picture descriptions with very specific stimuli, to mention just two research paradigms, are tasks that are only indirectly related to naturalistic (school) language tasks. Evidence from such studies is scientifically relevant and can contribute to theory development, but with respect to pedagogical practices or policy recommendations they represent surrogate outcomes (Y’ in Figure 1). They have limited to zero relevance for an argument about literacy acquisition in multilinguals: The relationship between the psycholinguistic investigation of lexical access in bilinguals and policy recommendations regarding HL instruction is about as tenuous as the relationship between testing the effects of a molecule on cancer cells in a petri dish compared to the effect of a cancer therapy on survival rates of cancer patients (see Sutherland, 2013, for other examples of the surrogate outcome fallacy). Robust findings from language-use patterns as those practiced in school settings are needed.

3.2. Lack of robustness: Methodological shortcomings

Often, the quality of research into interdependence is mixed. From the point of view of theory testing, the gold standard would be research that randomly assigns learners to an experimental group where L1 literacy is taught effectively and to a control group where some other cognitively interesting activity not related to the L1s is carried out. The effects of such an intervention then would be measured over a period of several years since learning literacy takes several years. Such research is not available, for obvious reasons: It would be highly problematic to randomly select young learners and force them either to learn their HL or inversely to prohibit this learning for others. Thus, research on HL learning is usually based on survey data of self-selected samples of HL earners (cross-sectional or longitudinal).

Many scholars find positive linear associations between skills in L1s and L2s (see Berthele & Lambelet, 2017b, for an overview), and such findings are then interpreted as evidence for interdependence, especially if the learners with HL instruction outperform the learners without HL instruction. Given that the groups getting HL instruction are overwhelmingly self-selected samples, such studies are biased towards confirmation of the interdependence theory: The phenomenon is studied “in winners” (Goldacre, 2009, chapter 11). What is interpreted as a positive effect of HL instruction (on HL or on L2) is at least in part the effect of a sample where the not-so-linguistically gifted are underrepresented. Ignoring this leads to naïve statements (see example 1).

3.3. Theoretically expected direction of transfer effects

As studies often did not yield the expected positive associations, initial, simpler versions of the theory were reformulated as in (3).

(3) To the extent that instruction in Lx is effective in promoting proficiency in Lx, transfer of this proficiency to Ly will occur provided there is adequate exposure to Ly (either in school or environment) and adequate motivation to learn Ly. (Cummins, 1996, p. 111)

In even later developments, many scholars assume that there is two-way transfer across languages. Moreover, as discussed in Hulstijn (2015, p. 117), several variants of threshold theories are added to the idea of positive transfer in literacy skills, sometimes involving minimal levels in the L1, sometimes involving minimal levels in the L2 (e.g., Alderson, 1992). Such thresholds are hardly ever established a priori (i.e., before the data are analysed) but developed a posteriori: Once differences in the strength of linear cross-linguistic associations of skills are found, the threshold explanation is used to explain the pattern (see discussion below and Takakuwa, 2005, for a critical analysis of threshold theories). Cummins (1979, p. 230) argues that thresholds vary within and across individuals, which poses similar problems for policy recommendations as the case study perspective of DST discussed above.

Modifying the interdependence theory by adding elements such as unspecific thresholds or vaguely spelled out contextual conditions required for successful learning can lead to problematic uses of empirical findings: Explaining results post-hoc invoking such elements can be considered HARKING (hypothesizing after the results are known, Kerr, 1998). Instead of testing a hypothesis using data, we sometimes use data to find hypotheses which then are unsurprisingly consistent with these same data. Of course, this is by no means to say that we should never revise our theories based on unexpected patterns in our data. In the context of threshold theories, as an example, an interesting way to go could be to infer possible thresholds from patterns in the data and then put these thresholds to the test with new data.

Experimental evidence for the influence of learning one language on learning another is scarce, for the reasons spelled out above. In our longitudinal project on Portuguese as a heritage language, we therefore tried to come up with predictions based on the interdependence idea that are testable. We gathered data from a sample of Portuguese children in both French- and German-speaking Switzerland. One of our predictions was that literacy, which is developed in the L2 in a much more intensive way than in the HL, would be transferred to the HL in a cross-lagged pattern. That same optimistic take on inter-lingual transfer of skills had shaped my view expressed in the newspaper interview (1). However, our data show that although skills measures across languages and within languages were all positively correlated, there is no statistically significant larger effect from L2 to HL than in the other direction (for details on these analyses see Berthele & Vanhove, 2017).

There are several possible patterns in correlational data gathered in the HL and SL context:

1. a negative linear association of HL and L2 skills;
2. no association between HL and L2 skills;
3. a positive linear association of HL and L2 skills;
4. non-linear associations (modifying patterns A or C, e.g., due to thresholds)

In short, a multitude of data patterns are compatible with the interdependence plus threshold theory: If there is no association or if it is negative, then scholars can argue that the necessary threshold has not been reached in one or both languages or that the learners lacked motivation to learn the HL. If the association is non-linear, then this can be interpreted as evidence for thresholds. If it is positive and linear, this can be considered as evidence of interdependence in a more straightforward way. However, positive correlations must not be confounded with causality since they can be due to all kinds of other effects, ranging from socio-economic correlates to task-wiseness to general cognitive abilities that are not related to language exclusively. Many influential scholars in the field do not advocate a strict separation between linguistic skills and other cognitive attributes. This is certainly true for most of the usage-based approaches to language learning (e.g., Wulff & Ellis, 2018) but also for interdependence-related research (e.g., “transfer and [cognitive] attributes are two sides of the same coin.” Cummins, 2017, p. 107). Such a fused construal of linguistic and cognitive skills, in my view, is theoretically and empirically sound. However, it raises important questions regarding the policy recommendation that is at stake: If the common proficiency underlying the claim that investments in the HL are good for the L2 is (largely?) formed by general cognitive skills (memory, fluid intelligence, etc.), then it becomes rather unclear why investing in the learning of one language should be particularly beneficial for another language. To the extent that these general cognitive skills can be trained and transferred, any cognitively interesting activity should therefore be shown to have beneficial effects on literacy acquisition in the L2.

In sum, as others have argued before (e.g., Hulstijn, 2015, p. 130), we are dealing with a theory that has many flaws. In my view its main problem is that it intuitively makes sense while being too powerful: It is true regardless of what the data say. A theory that is consistent with any imaginable type of finding pertains to pseudoscience, resembling astrology (Pigliucci & Boudry, 2013, p. 16, Figure 1.1). More importantly, unless the thresholds or the contextual conditions and proficiency levels required for transfer are specified, no recommendations should be given to policy-makers, since the effect of X→Y is one among several possible consequences of HL instruction (Z) occurring under certain ill-specified conditions.

3.4. Interdependence: A discursively successful scholarly failure

Along the lines laid out in Figure 1, we can thus conclude that the evidence for X→Y is inconclusive, although many scholars find significant positive associations of skills across the two languages.

The methodological shortcomings of the research make it impossible to ascertain causal influences from HL literacy skills to L2s (or vice versa, for that matter). The interdependence approach, if supplemented post hoc by threshold explanations, is vague and does not lend itself to hypothesis testing and to the advancement of theory. Given this situation, research on Z (i.e., on the impact of HL instruction on X (which in turn influences Y)) is superfluous if its goal is to put the whole theory Z→(X→Y) to the test. In our recent book (Berthele & Lambelet, 2017a), several studies on HL and L2 development were presented, and the methodologically most compelling is a study by Moser et al. (2017). This quasi-experimental, longitudinal study potentially allows statements about Z→(X→Y) because it compares skills from a control group and an experimental group, and only this experimental group had followed a carefully designed pedagogical intervention in the respective HLs. The study, however, did not even find an effect of HL instruction on HL proficiency. Of course, this is only one study, but other intervention studies that test the effects of pedagogical measures fostering bilinguals’ languages find similar results (e.g., McElvany et al., 2017). As is often the case, the higher the quality of the empirical investigation, the smaller the effects of the supposed beneficial measure.

To sum up, the evidence for causal transfer effects from L1 to L2 in the literacy domain is inconclusive (the effects are spurious, the causality behind the linear association is unclear), and the evidence for positive effects of HL instruction on L2 is scarce. Recommendations of HL instruction based on this evidence, with the rationale that proficiency in HL will foster proficiency in L2, thus seem to lack an adequate empirical base. The notion of linguistic interdependence is above all used to argue in favor of bilingual education, in contexts where speakers of particular HLs form a large proportion of the pupil population. Of course, bilingual education and various forms of HL instruction can be considered necessary or good for other reasons, and therefore research on HL instruction may still be relevant (I will come back to this in the concluding section). But ideas of transfer-benefits based on the interdependence theory are currently not supported by robust evidence.

4.1. Lack of fit: Surrogate outcomes

As mentioned above, the fundamental problem with the debate on AoO in FL instruction is that it mainly draws on evidence from L2 acquisition. Most colleagues, regardless of their theoretical affiliation, would agree that there are age effects if one examines cohorts of L2 learners, and that these effects, regarding proficiency in the long run, can be grossly summarized as ‘the younger the better’. However, such studies focus on surrogate outcomes when FL rather than L2 learning is at stake. In the Swiss case as in many others, discussion also drew on neurolinguistic evidence concerning the activation patterns of younger versus later bilinguals, with an earlier AoO yielding more overlapping activation in specific areas of the brain than later AoO (see, for example, Imgrund & Le Pape Racine, 2005). The study often cited in this debate, however, did not focus on proficiency but on age-related differences in the locus of brain activity; proficiency in earlier and later learned languages was held stable (Wattendorf, 2006).⁴ The main researcher of the study, the neurologist Elise Wattendorf, never drew any curriculum-oriented inferences regarding what the different age-related activation patterns mean (see Berthele, 2014, for a discussion). These brain-activation patterns, interesting as they may be for brain researchers, are not even a surrogate outcome here and cannot serve as a basis for curriculum planning. Instead of critically reviewing the research on the effects of AoO in FL teaching, many scholar-advocates enriched their publications by irrelevant references to brain research, which brings to mind McCabe and Castel’s study (2008) showing that research reports including irrelevant information on brain research are considered more convincing by readers.

What is needed to decide whether an earlier start of FL teaching is beneficial or not is again experimental, or at least quasi-experimental, research that compares maximally comparable cohorts of learners that are different with respect to AoO.

As to the idea of transfer from the first to the second FL, there is indeed evidence from bi- and multilingualism research that learners transfer in many different directions (Pavlenko & Jarvis, 2002). Although no explicit references are given, the recommendation in (5) is most likely influenced by holistic and transfer-oriented theories of the multilingual repertoire, as is a large body of the programmatic literature on multilingual language learning (see the discussion in section 3 and the chapters in Hufeisen & Neuner, 2004, for further illustration). From our own studies on receptive skills in multilinguals, I am aware of a long-standing interest in spontaneous positive transfer of one language in the repertoire to another (see Vanhove & Berthele, 2015, for references). Studies from bilingual contexts yield mixed results: Sometimes bilinguals outperform monolinguals in L3 learning and sometimes they do not (cf. Cenoz, 2003, for an overview). However, for the Swiss curriculum reform in question, such studies are again surrogate outcomes in the sense that the proficiency acquired after two years of 2–4 weekly lessons in, say, French as a FL cannot be expected to be on a par as a basis for transfer with one of the two languages mastered by early bilinguals in Basque and Spanish in the Basque country. Also, the spontaneous positive transfer researchers in receptive multilingualism generally observe is often investigated in adults, and, even worse, in undergraduate students of linguistics or philology. Inferences from such gifted language users and learners to the population of children in compulsory primary school are risky, and it is a general feature of the programmatic literature on (early) FL learning that it is mainly informed by the introspection of language experts and by surrogate research outcomes from badly fitting samples of multilinguals.

4.2. Lack of robustness: cherry picking, CARKing and HARKing

There is one study of transfer from the first to the second FL that is often cited in the Swiss context (Haenni Hoti et al., 2011). This study, in my view, is a well-designed comparison between pupils from cantons that, at the time of data collection, just had or had not yet introduced two FLs at primary level. The study found better skills in French (as a second versus first FL) listening and reading at the end of the fifth grade. This is often taken as proof for such a beneficial transfer effect from one to the other FL. An extension of this study measured the same effects once more one year later. By then, the positive effects of French as a first FL on English as a second FL in listening and reading comprehension were gone (Heinzmann et al., 2009, p. 45ff.). This extension study, however, is unmentioned in important reviews. For example, in the systematic review by Dyssegaard et al. (2015, p. 81), only the results of the first publication 2011 are discussed.⁵ Or, if the extension evidence is discussed, CARKING (criticizing after the results are known, cf. Nosek & Lakens, 2016) is practiced, in this case by the authors themselves: They criticize the quality of their own tests in the face of the null result, asserting that better tests would have yielded the expected result.⁶

Manno (2017) presents another highly relevant study, in a similar context immediately before and after the transition to two FLs in primary school. He shows no positive effects of the first FL (English) on the second FL in reading skills (French) if the French skills are compared to the group that only learns French as a FL. In the concluding discussion, the author comes up with post hoc threshold explanations (Manno, p. 147) that could account for the null result. Had the prediction been that a specific level in the first FL needs to be attained before transfer to the second FL occurs, this prediction could have been put to the test. But testing the effect of skills in a first FL on a second FL and, in the absence of the expected result, postulating thresholds is a case of HARKING and does not contribute to the advancement of the discipline, let alone license any policy recommendations. As in the interdependence case, the theory seems to predict that skills are correlated unless they are not.

As we discuss in Lambelet and Berthele (2015), even if we could design an experimental study of AoO differences that would run for several years, there would still be the question of what we should compare: Learners of different ages after the same exposure time? Learners of the same age with different exposure time? Moreover, the question arises whether it is possible to use identical language tests for learners of different ages or who have been taught with different, age-adapted methods. Despite the lack of large-scale, robust evidence, the studies that are available of AoO and FL learning are far from clearly supporting the earlier onset in the sense of the converging evidence.

Like studies in the field of HL speakers, studies in the FL field are characterized by vague theories and an optimistic view of language teaching and learning. There is a danger that pedagogical reforms based on firm beliefs but shaky evidence will harm the vulnerable while the gifted learners are likely to learn even in the most adventurous pedagogical paradigms (cf. Berthele et al., 2017 on potential Matthew effects in multilingual language teaching; also Willingham, 2012, p. 16).