Pedagogic natural language processing resources for L2 education: Teachers’ perceptions and beliefs

Carlos Ordoñana-Guillamón

carlos.ordonana@cud.upct.es

Centro Universitario de la Defensa – CUD San Javier, Spain

Pascual Pérez-Paredes

pascualf@um.es

Universidad de Murcia, Spain

Pilar Aguado-Jiménez

paguado@um.es

Universidad de Murcia, Spain

ABSTRACT

Pedagogic natural language processing resources (P-NLPRs) are a group of online technologies that aid teaching practices and hold the potential to enable Data-Driven Learning approaches by providing teachers and students with linguistic information. This study explores the perspectives of L2 educators on the potential implementation of P-NLPRs in their teaching practices. A training module was designed to provide information on the potential applications of different P-NLPRs, from which quantitative data was gathered (n=77) from PRE- and POST-surveys. Additionally, individual interviews were carried out with some of the participants (n=4) five years later to assess long-term P-NLPR uptake. Results offer insight into educators’ perception towards adopting P-NLPRs for their language teaching. Their perspectives seem to differentiate three main groups: a) tools to help learners learn (i.e. online dictionaries, text-to-speech technologies); b) tools to help teachers teach (i.e. automatic summarization tools, lexical profilers), and c) tools to help expand linguistic knowledge (corpora, POS taggers, lemmatizers).

Keywords: Data-Driven Learning; new literacies; teacher training; language processing technologies; individualized learning

I. INTRODUCTION

Data-Driven Learning, or DDL (Johns, 1991), aims to foster the personalization and individualization of language learning (Pérez-Paredes et al., 2019) by encouraging learners to take an active role when discovering linguistic patterns, generally through the manipulation and exploration of textual data (Boulton & Vyatkina, 2021; Farr & Hagen Karlsen, 2022). DDL has garnered increasing attention from scholars in the last decade (Boulton & Vyatkina, 2021), as the literature has suggested that DDL promotes problem-solving, analytical skills, autonomy, and language awareness in learners (Farr & Hagen Karlsen, 2022). Scholars have pointed out that this approach can only be viable across different educational levels if it is adopted by ‘ordinary‘ members of the community (Pérez-Paredes, 2010). In other words, teachers and learners outside tertiary education need to engage with language processing tools before they can be perceived as useful in L2 classrooms (Chambers, 2019).

However, in spite of the general consensus among scholars about the benefits of DDL approaches, the area seems to struggle in achieving a widespread uptake both among academics, and students and educators (Chambers, 2019; Pérez-Paredes, 2022). Consequently, recent literature has started to suggest that DDL would benefit from a ‘revamp’ in terms of re-assessing how to approach DDL research as well as how to best tackle the practical implementation of its methods (Meunier, 2020, 2022). This includes the possibility of expanding the potential toolset of DDL beyond corpus-based resources, taking hold of the wide range of digital tools available on the internet, including Natural Language Processing Technologies.

Natural Language Processing Technologies (NLPTs) are technologies that process, generate and manipulate written and spoken language with the use of software applications for different purposes (Nilsson, 2009). They are available on desktop and mobile devices (Bird et al., 2009; Pérez-Paredes et al., 2018), either via desktop applications and web browsers or integrated in mobile apps, respectively. The use of NLP tools in education contexts has been extensively studied during the last decade (Zhu, 2023); NLPTs such as automatic speech recognition, word frequency count, vocabulary profiling, online lexical databases and dictionaries have shown positive results in improving pronunciation (Golonka et al., 2012), lexical retention (Cobb, 1997), translation skills (Mekheimer, 2012), reading level assessment for L2 students (Huang & Liou, 2007) and in increasing L2 writing awareness (Pérez-Paredes et al., 2019).

The connection between NLPTs and DDL is by no means new to academia, as Cantos (2002) and Granger et al. (2007) had already suggested using natural language processing tools as a way of promoting DDL-like practices. Pérez-Paredes et al. (2018, 2019) explored the perspectives of teachers and learners from different backgrounds towards using various NLP tools as vehicles of DDL to promote learners’ linguistic awareness through the autonomous interaction with the target language. However, academia seems mostly oblivious regarding this connection, as direct research addressing the combination of NLP resources and DDL are scarce.

Therefore, this paper aims to address the gaps identified in Meunier (2020, 2022) and Pérez-Paredes et al. (2018, 2019) by proposing pedagogic NLPT resources (P-NLPRs) as a group of technologies that process both L1 and L2 languages in order to facilitate L2 learning across levels and educational contexts. P-NLPRs may provide language teachers and learners with opportunities to personalize the L2 learning experience. These resources include, among others, online dictionaries, automatic text-to-speech tools, and lexical profilers (see Figure 2).

The present study hence provides a preliminary exploration on the potential impact of P-NLPRs in education contexts, stressing the perspectives of in-service teachers after completing a professional development initiative in the context of an Erasmus+ project that aimed to promote the use of P-NLPRs. Using a mixed-methods research design (Teddlie & Tashakkori, 2009), we look at the perspectives of a group of language teachers (n=77) that completed the course and individual interviews (n=4) that revisited the take-up and use of P-NLPRs five years after the completion of the course. Our paper seeks to contribute to understanding how NLP technologies can inform language teaching practice and how they can favour personalized language learning across the board. Additionally, this research addresses gaps in the literature by exploring the perspectives of in-service language teachers from different countries with varying degrees of teaching experience

The following sections will provide the rationale behind our research questions, as well as a detailed description of the different elements that are part of our study design.

II. LITERATURE REVIEW

II.1. Digital Literacies and P-NLPRs in L2 Teaching Practices

Digital literacies, “a constellation of symbolically mediated practices that involve various kinds of knowledge, predispositions, and skills to deal with texts in electronically-mediated environments” (Kern, 2021; p.134) do not refer solely to the handling of electronic devices, but rather to an individual’s agency to navigate digital environments (Elola & Oskoz, 2017). Digital literacies allow for a more autonomous, agentive role for learners (Lotherington & Ronda, 2014) and educators as digital literacies can provide individualized learning experiences (Chapelle, 2006). The efficient use digital resources such as P-NLPRs demands from users a set of skills based on the adequate management and interpretation of increasingly sophisticated symbolic systems (Kern, 2021). The acquisition of such a skillset is important for both instructors and learners alike, as the former need to be able to guide their students in technology-mediated contexts. The TPACK model (Technological Pedagogical Content Knowledge) aims to address such challenges by facilitating teaching development programmes to provide learning opportunities to develop teachers’ technological knowledge (TK), pedagogical knowledge (PK) and content knowledge (CK) on the use of different technological resources (Cabero & Barroso, 2016; Mishra & Koehler, 2006). This has become even more essential in a post COVID-19 world, especially in the context of language learning, as traditional education has been forced to adapt to online environments (Kern, 2021).

Teacher training on technology usage has received great attention from CALL research. Scholars have also noted that sustained and supported “opportunities to learn something new or to learn about something familiar more deeply” should be “grounded in immediate teaching contexts [and] encouragement to change classroom and school practices in innovative ways” (Knobel & Kalman, 2016: p.3). Romeo & Hubbard (2010) found that providing information on how to use different applications related to listening skills resulted in learners increasing their ability to autonomously engage with the technologies for listening practice. A similar course of action has been suggested with Open Educational Resources (OERs) (Littlejohn & Hood, 2017). Trust, Maloy, and Edwards (2022) designed a training module aimed at posing students as OER designers and curators, with promising results in increasing student motivation, attitudes towards learning and supporting the development of new literacies’ skills. Projects such as CATAPULT (http://catapult-project.eu/) aim to train language for specific purposes (LSP) teachers, providing them with the necessary resources to, in turn, teach their LSP students. Further research emphasizes the importance as well of fostering authentic experiences, hands on practice and teacher reflection in ICT teacher training (Hsu & Lin, 2020).

Scholars point out that specialized training becomes even more crucial when dealing with corpora and DDL-based instruction. Chambers (2019) warned not only against a lack of DDL-proficient educators, but also against a scarcity of DDL experts in academic institutions able to provide the appropriate training. Training in corpora and DDL-based literacy has extensively been called upon by scholars; Poole (2020) suggested that users might find corpus-based methodologies as ‘inefficient’ due to difficulties in managing corpus data and corpus feedback. Leńko-Szymańska (2017) pointed out that DDL-specific training should promote teacher autonomy and enable them to design DDL-based materials tailored to student needs. Recent efforts in providing corpus literacy instruction to pre-service teachers include Le Foll (2024) and Abdel-Latif (2021)

The question of how to approach proficiency building in technology has been thoroughly explored by the literature. Several well-known theoretical models aim to represent user behaviour in technology acceptance and uptake, such as the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al., 2003), and the Technology Acceptance Model (TAM) (Davis, 1989) and its updated version, TAM 2 (Venkatesh & Davis, 2000). These frameworks have been commonly used to evaluate and predict the uptake of emerging technologies (Criollo-C et al., 2024). Among the most important factors that influence potential behaviour, intention is regarded as an important predictor of behaviour, which in turn is influenced by other factors such as performance expectancy or perceived usefulness, that is, an individual’s belief that a certain system can improve their job performance (Davis, 1989; Venkatesh et al., 2003).

Perceived usefulness has been extensively studied in relation to language learning technologies (Lai, 2013) and OERs (Kelly, 2014). Research has suggested that teachers tend to make use of the technologies they are more comfortable with due to using them on a daily basis (Kessler, 2018). It has also been suggested that increased familiarity with a given product is linked to its perceived usefulness (Zhu & Chang, 2015). Pérez-Paredes et al. (2018), and Ordoñana et al. (2018) have argued that familiarity with P-NLPRs could in fact be a main predictor of a further increase in frequency of use. Thus, increasing teacher knowledge of how P-NLPRs work to foster their familiarity with P-NLPRs could in turn increase their frequency of use of these resources. However, a generalized lack of familiarity with P-NLPRs has been reported (Pérez-Paredes et al., 2018), even though research shows there is a growing interest among educators regarding the possibilities OERs and P-NLPRs may offer (Farrow et al., 2015; Pérez-Paredes et al., 2018). The literature has called for increased efforts in disseminating OER and P-NLPR knowledge to foster familiarity with the resources (Pérez-Paredes et al., 2018), which additionally remains in line with the necessity to keep up with the constant development of new literacies (Kern, 2021; Trust et al., 2022).

As stated above, the present paper aims to follow up on the gaps previously identified in the literature: addressing the issues with DDL requires an attempt to ‘revamp’ the area by opening up the potential toolset to other resources beyond using only corpora (Chambers, 2019; Meunier, 2020, 2022); yet, the actual adoption of most of the tools included among P-NLPRs is fairly limited (Pérez-Paredes et al., 2018, 2019), which hurdles the assessing of the viability of P-NLPRs as vehicles for DDL practices and their impact in promoting individualized and personalized learning. Hence, the present study will first describe the process of implementation of a training module specifically designed to target the perceived usefulness of P-NLPRs as vehicles for DDL within the framework of the Transforming European Learner Language into Learning Opportunities (TELL-OP) project.

Thus, our research questions (RQ) aim to discuss whether our efforts in building up knowledge and long-term usage have been successful: (1) Has the TELL-OP training module in the use of P-NLPRs succeeded in contributing to the promotion of language teachers´ perceived usefulness of P-NLPRs?

(2) After five years of hands-on experience, how do in-service teachers perceive the viability of using P-NLPRs in their respective educative contexts?

(3) According to in-service L2 teachers, how can NLP tools favour personalized and individualized learning across the board?

II.2. Materials and Methodology

This study uses a sequential mixed-methods (Teddlie & Tashakkori, 2009) one-group PRE-survey/POST-survey repeated measures design without a control group, followed by a series of long-term interviews. Repeated measures were applied to several scales drawn from related items from a questionnaire developed specifically for this study. In the following paragraphs we will offer a description of the training module the subjects participated in, as well as detailed information on the data collection and analysis.

The present study is thus designed to provide both qualitative and quantitative insights on the perspectives of in-service L2 instructors towards using P-NLPRs in their teaching practices.

II.2.1. The Sample

Second language teachers from across the EU were invited to participate in the course (see section II.2.2.) through social media, newsletters, and e-mails. The recruitment strategy included dissemination of information throughout university departments, secondary education institutions, and teacher Unions. All participants were second or foreign language educators. Among those who started the program, 345 completed an initial PRE-survey questionnaire, and 39.1% (n=135) completed a POST-survey questionnaire. To ensure that results could be fully matched and compared between the PRE- and POST-surveys, each participant was assigned a personal code to be included at the beginning of both questionnaires. Finally, after completing the course, 77 subjects filled in both questionnaires with the individual code and could be adequately matched for analysis. The comparisons reported in this research are based on the responses from those 77 subjects.

Most participants were female (92.2%) and over 30 years old (44.2%). The most widely represented countries were Germany (37.7%), Belgium (31.2%) and Spain (14.3%), while the additional 17% grouped teachers from the UK, France, Austria, Italy, Poland, Macedonia, Ukraine, or Kazakhstan, and even four subjects from the US and Australia. Their teaching was similarly distributed between Secondary (31.2%), Higher (26.0%) and Adult education (31.2%). Finally, 48.5% had less than 5 years of teaching experience, 34.9% between 10 and 15 years, and 16.7% more than 15 years.

II.2.2. The Training Module

The module was developed as part of the Erasmus+ project TELL-OP, a strategic partnership that aimed to promote DDL practices using ICTs and OERs (TELL-OP, 2015). The project included five teams of language learning experts from different countries (United Kingdom, Belgium, Germany, Spain and Turkey). This module aimed to promote TPACK-based knowledge (Mishra & Koehler, 2006) about different P-NLPRs available online in order to foster the potential adoption of these tools for the participants’ teaching practices by providing a detailed description of the different types of tools and their potential application in a language education context. The course was set up on a Moodle platform (https://moodle.org/) and took 25 hours in the timespan of five weeks, from January 16th, 2017, to February 17th, 2017.

Table 1. A sample of some of the P-NLPRs included in the module. Note that some of them may not be available as of the time of the reading of this article

Text-to-Speech technologies	http://www.naturalreaders.com/index.html http://text-to-speech.imtranslator.net http://www.fromtexttospeech.com https://text-to-speech-demo.mybluemix.net
Lemmatizers	http://textanalysisonline.com/nltk-wordnet-word-lemmatizer http://textanalysisonline.com/nltk-wordnet-lemmatizer
Resource-linked text builders	http://lextutor.ca/ra_read/ http://www.lextutor.ca/hyp/ http://sifnos.sfs.uni-tuebingen.de/VIEW/
Text summarization tools	http://autosummarizer.com/ http://freesummarizer.com/ http://textsummarization.net/text-summarizer http://www.splitbrain.org/services/ots http://textcompactor.com/
Online dictionaries and collocation dictionaries	http://www.oxforddictionaries.com/ http://www.oxfordlearnersdictionaries.com/ http://en.pons.com/translate http://dict.leo.org/ende/index_en.html http://forbetterenglish.com/index.cgi http://www.linguee.com/
Automated POS taggers	http://nlp.stanford.edu/software/tagger.shtml http://ucrel.lancs.ac.uk/claws/trial.html http://parts-of-speech.info https://gate.ac.uk/wiki/twitter-postagger.html
Lexical profilers	http://www.lextutor.ca/vp/eng/ http://www4.caes.hku.hk/vocabulary/profile.htm http://www.sfu.ca/~msevier/WebVocabularyProfilerCS.htm http://vocabkitchen.com http://www.insightin.com/vocabulary/profiler.php
Word lists by frequency count	http://www.wordfrequency.info http://www.writewords.org.uk/word_count.asp http://www.wordcounter.com http://www.textfixer.com/tools/online-word-counter.php http://www.online-utility.org/text/analyzer.jsp
Social networking services	https://www.pinterest.com https://www.tumblr.com https://www.instagram.com https://www.snapchat.com https://www.flickr.com https://www.podomatic.com/ https://www.facebook.com/ https://twitter.com

The contents of the module were designed following standard practices in online training (Vai & Sosulski, 2015). Learning materials were grouped under five main blocks: (1) Pronunciation; (2) Vocabulary Acquisition; (3) Reading Skills; (4) Interaction; and (5) Writing Skills. Each block contained information on several related P-NLPRs, which was in turn structured into four sections: (a) what is it? (b) how can it be used? (c) how does it contribute to language learning? and (4) P-NLPR examples and P-NLPR practice. Each block was made available weekly to participants over five consecutive weeks. For each module, participants were encouraged to take part in forum discussions and assignments related to the tools suggested in each block. The assignments prompted language teachers to use some of the P-NLPRs from the block to gain hands-on experience. They were then encouraged to write an account of their experience with P-NLPRs.

The selection process of the tools for the module followed specific criteria: the P-NLPRs included had to be available via a computer or a mobile device during the period of the training module for free and to serve a pedagogical purpose for language learning. All the selected P-NLPRs targeted English, although some of them worked for other languages such as Spanish or German. Table 1 shows examples of the selected P-NLPRs. The module was delivered in four different languages, English, Spanish, German and French, and was officially offered in the United Kingdom, Spain, Germany and Belgium. The enrolment and certification process were coordinated by the four participating universities.

II.2.3. PRE-survey/POST-survey: The Questionnaires

The quantitative data were collected using 5-point LIKERT-scale surveys (Owen, 2017) with a PRE-survey/POST-survey design to compare the results prior to and following completion of the module. Items from both PRE-survey and POST-survey were designed and selected following notions of theoretical pluralism (Stockwell, 2022). The items in both questionnaires were designed to target the subjects’ previous knowledge of the tools (i.e. whether the participants know what the tools are and what they can do) (PRE-survey) as well as knowledge acquired after the course (POST-survey), familiarity (i.e. how accustomed the subjects were to using the tools on a frequent basis) with the different P-NLPRs before and after the module and perceived usefulness of the tools.

The PRE-survey comprised 49 items including demographic and professional information (age, country, years of experience and type of institution where they are employed) as well as specific questions and scales on language teachers´ knowledge, experience, perception on the role of P-NLPRs on teaching practices, familiarity with P-NLPRs, and frequency of use of P-NLPRs prior to the course. The POST-survey consisted of 24 items, which dealt with the participant’s engagement with the course contents, knowledge of P-NLPRs, intention to use them in the future, and scales on the role of perception, familiarity and perceived usefulness of the P-NLPRs (Table 2).

Table 2. Main evaluation variables in PRE- and POST-survey

		PRE-survey	POST-survey
Knowledge of P-NLPRs		✓	✓
Experience with P-NLPRs		✓
Intention to use P-NLPRs in the future			✓
Opinion of course			✓
Scales	Perceptions of P-NLPRs	✓	✓
	Familiarity with P-NLPRs	✓	✓
	Frequency of use of P-NLPRs	✓
	Perceived usefulness of P-NLPRs		✓

Knowledge of P-NLPRs, previous experience and intention to use P-NLPRs in the future were measured through a single item each. Engagement with the course included three items assessed individually (whether the course had been useful, showed new ways of teaching, and increased willingness to use P-NLPRs). Scales on perceptions and familiarity with P-NLPRs on the PRE- and POST-surveys included 5 and 15 items respectively. The scale on the perceptions on the role of P-NLPRs for teaching included items evaluating their initial, subjective view of the general role that P-NLPRs may have as a tool for improving language teaching practices (e.g., whether they were easy to integrate, student appreciation of their use, or their utility for reaching out more students). The familiarity scale covered the different P-NLPRs individually in order to assess how successful the module was in increasing participants’ familiarity with them. Likewise, the perceived usefulness scale comprised the same item list and measured the extent to which each P-NLPR is considered useful (e.g. “I think online dictionaries are useful for my language teaching”). Response for all evaluative items ranges between 1 (Not at all/Completely disagree) and 5 (Always/Completely agree). Item scores were averaged within each scale to ensure they kept within range.

Both questionnaires were developed specifically for this study and were checked for content validity by several experts participating in the TELL-OP project. Inventory of instruments included in the list for evaluation of familiarity, frequency of use and perceived usefulness of the different P-NLPRs were selected from suggestions of an international expert panel. In the case of the perceptions on the role of P-NLPRs scale, construct validity was tested by means of confirmatory factor analysis (CFA) to confirm its unifactorial structure [RMSEA= 0.016, 90%CI 0.00-0.08; CFI=0.99; TLI=0.99). Reliability tests were carried out on each scale, using the whole sample of the PRE-survey (n=345), or POST-survey (n=135) in the case of the perceived usefulness scale. Scale reliability was assessed through Cronbach’s alpha (Perceptions on the role of P-NLPRs = .786; Familiarity with P-NLPRs = .896; Frequency of use of P-NLPRs = .903; Perceived usefulness of P-NLPRs = .950)

II.2.4. The Interviews

The questions for the interview were designed ad hoc for this study. The interview was conducted in Spanish, structured into four distinct parts. Firstly, the participants were given, as a reminder, an overview of the different P-NLPRs covered in the module. They were also asked how frequently they had used them after the course and why. Secondly, they were inquired about the possible impact of the tools in their teaching practices and the role of authentic language in class. Thirdly, they were asked about DDL and the impact of these tools in the approach. The final part of the interview dealt with their perspectives on individualized learning. The script the interviewer used to guide the data collection can be found among the supplementary materials accompanying this work. The sessions took place via ZOOM and lasted between 40 and 100 minutes each, resulting in 39,488 words transcribed. The transcripts were then coded using a deductive approach (Azungah, 2018) in combination with grounded theory (Strauss & Corbin, 1994). Each of the interviewees’ responses was analysed in order to extract the main ideas, which were later grouped and categorized to, in turn, allow for a more comprehensive, qualitative analysis (e.g. see table 7 below)

II.2.5. Statistical Analyses

Paired-sample t-tests were performed to compare pre and post intervention scores for variables with repeated measures (P-NLPR knowledge, perception and familiarity). Additionally, multiple regression analyses were carried out on two continuous variables that were considered as main outcomes: perceived usefulness and intention to use P-NLPRs following intervention. Univariate linear/ordinal regression analyses were first conducted to assess the association between each predictor variable with the outcomes in order to explore individual relationships between them (Cohen et al., 2003; Pedhazur, 1997). Variables which showed significant associations in the univariate analyses were then entered in multivariate linear regression models to determine whether the relationship was independent and strong enough to remain significant in the presence of other variables and which presented the most relevant contribution to outcome prediction. All statistical analyses were performed with SPSS 28.0 for Windows (SPSS; Chicago, IL).

III. RESULTS AND DISCUSSION

III.1. Results

In the present section we describe the outcomes of the different analyses performed for the present study. Subsections III.1.1, III.1.2 and III.1.3 will outline the results of the quantitative analyses, which will provide the necessary insight to address RQs 1 and 2. Subsection 4.4 will outline the data gathered in the qualitative analysis, which will in turn be used as the basis for the discussion in RQ3. First, we will describe the participants’ relationship with P-NLPRs; that is, their knowledge, experience and perceptions regarding these kinds of teaching resources (III.1.1). We also examine whether the course had an impact on the participants by comparing the responses obtained after the course to the PRE-survey scores on those variables that were measured at both moments (Knowledge, perceptions and familiarity regarding P-NLPRs) (III.1.2). We continue by determining, by means of regression analyses, which of the measured variables showed a stronger association with the main outcome variables assessed in the POST-survey (i.e., Intention to use and perceived usefulness), which are thought to influence actual use of P-NLPRs (III.1.3). These outcomes were further checked using the magnitude of the changes produced during the module as predictors to ascertain the impact of the course progresses on intention and perceived usefulness. Long-term perceptions of the course participants are described qualitatively in (III.1.4).

III.1.1. Language teachers’ engagement with the module and P-NLPRs

The subjects declared (Table 3) having moderate previous knowledge of P-NLPRs (M=2.39; SD=1.2) and experience in using them (M=2.11; SD=1.27). However, their perceptions of the role of P-NLPRs in their language teaching outscore their actual practice with greater means for all items in the scale, including expectancies regarding students and the institution´s appreciation of incorporating P-NLPRs to language teaching. Nevertheless, the general frequency of use was quite low (M=1.85; SD=0.63), and only “use of online and collocation dictionaries” showed a score above 3 (M=3.64; SD=1.37).

Table 3. Profile of the participants

ID	age range	mother tongue	gender	Qualifications	years of experience	level taught	based country	Language level (CEFR)
A	45-54	Polish	Female	MA	+20 years	Adult Education	Spain	C1
B	45-54	Spanish	Female	PhD	+20 years	Tertiary Education	Spain	B1, B2
C	45-54	Spanish	Female	MA	+20 years	Secondary Education, Tertiary Education	USA	A1, B1, B2
D	35-44	Spanish	Male	PhD	11-15 years	Tertiary Education	Spain	C1

In the POST-survey, results regarding module evaluation showed largely positive evaluations. The module was viewed as useful and capable of showing innovative ways of teaching. This apparently fuelled participants´ knowledge of P-NLPRs and their intention to use them, which reached a high score (M=3.9; SD=0.92 and M=4.11; SD=0.94, respectively). Hence, the training experience had seemingly produced changes in the participants ’ relationship with P-NLPRs, which we test and describe in the following section.

III.1.2. Language teachers’ trajectories during the module

Paired-sample t-tests were carried out to compare pre and post intervention scores with two main objectives: as a basic checking procedure of the appropriateness of the intervention and for determining the magnitude of the changes. Analyses showed that perception of P-NLPRs increased slightly and non-significantly [t(76)= -1.55; p= .125; MPRE= 3.65, SD=0.8; MPOST= 3.80, SD=0.8] after the course (Table 4; Figure 1). However, both familiarity with P-NLPRs[t(76)= -11.48; p= <.001; MPRE= 2.54, SD=0.8; MPOST= 3.85, SD=0.9] and P-NLPR knowledge [t(76)= -10.115; p= <.001; MPRE= 2.49, SD=1.1; MPOST= 3.90, SD=1.0] (Figure 1) significantly increased after completing the module.

Figure 2 displays the difference between scores at PRE- and POST-survey for items in the familiarity scale. The familiarity rate is higher for each item in POST-survey. Items which participants reported being more familiar with before completing the course, such as Online Dictionaries (PRE= 3.82), Social Networking Sites for Written Discourse (PRE= 3.73), and Spell Checkers (PRE= 3.39) are those which increase the least (POST=4.42, POST=4.25, and POST=4.21, respectively), while the least known items, namely Lemmatizers (PRE= 1.54), Resource-linked text builders (PRE= 1.62) or Automated POS taggers (PRE= 1.63) experience a starker increase (POST=3.59, POST=4.25, and POST=4.21, respectively).

Table 4. Mean (SD) scores for PRE- and POST-survey for different variables related to self-reported knowledge, experience, perceptions about P-NLPRs, intention to use them and opinions about the course. Items that composed the scale in italics

Item/Scale	PRE-survey	POST-survey
Engagement with the course
This course has been useful		4.33 (1.0)
This course has shown me new ways of using P-NLPRs in teaching		4.38 (0.9)
Knowing more about P-NLPRs increased my willingness to use them		4.27 (0.9)
I have knowledge of P-NLPRs (Knowledge)	2.39 (1.2)	3.9 (0.9)
I have experience using P-NLPRs (Previous experience)	2.11 (1.3)
Perception of P-NLPRs (Scale)	3.65 (0.8)	3.80 (0.8)
P-NLPRs may be useful for teaching languages	4.01 (1.0)	4.47 (0.8)
P-NLPRs help me to reach out to more students	3.40 (1.2)	3.68 (1.1)
I think P-NLPRs are easy to integrate into my daily teaching	3.17 (1.0)	3.39 (1.0)
I think my students appreciate my using P-NLPRs for language learning	3.41 (1.1)	3.87 (0.9)
I think my institution supports my using P-NLPRs for language teaching	3.26 (1.2)	3.67 (1.0)
Familiarity with P-NLPRs (Scale)	2.54 (0.8)	3.85 (0.9)
Perceived usefulness (Scale)		3.82 (0.8)
Intention to use P-NLPRs for teaching in the future		4.11 (0.9)

III.1.3. Perception of P-NLPRs usefulness and intention to use P-NLPRs following the module

Perception of usefulness and intention to use P-NLPRs were measured at POST-survey as the main outcome variables, as both variables are widely related to actual use of those teaching resources. Regression analyses were carried out for each of them independently. For each case, the rest of the measured variables were used as predictors. We first analysed the association of each single variable with the outcome (i.e., univariate analysis) to determine whether there was a significant association between predictor and outcome variables (Tables 5 and 6). Subsequently we analysed the joint effect of all the significant predictor variables on multivariate models, to ascertain which of the predictors showed a more robust and reliable association with the outcomes.

Table 5. Univariate linear regression analyses for perception of usefulness of P-NLPRs after the course. Bold text for significant results.

	B	CI 95%		Wald	P
Subject characteristics and working environment
Gender	1.281	-3.499	6.061	.276	.599
Age	-1.141	-4.057	1.776	.588	.443
Years of Experience	-.070	-1.009	.868	.022	.883
Working Institution
Secondary School	-3.605	-7.894	.685	2.712	.100
Higher Education	-1.846	-6.251	2.559	0.675	.411
Adult Education	-.852	-5.141	3.438	0.151	.697
Other (Ref)	0
Institution fosters the use of MDs	1.842	-.719	4.403	1.988	.159
MD training	3.201	.362	6.039	4.884	.027
P-NLPRs Training	1.833	-1.963	5.629	.896	.344
PRE-survey measures
Previous knowledge about P-NLPRs	-822	-.399	2.043	1.741	.187
Previous experience with P-NLPRs	.439	-.622	1.500	.658	.417
P-NLPRs Perception (PRE)	1.783	.142	3.423	4.538	.033
P-NLPRs Familiarity (PRE)	2.233	.789	3.677	9.185	.002
P-NLPRs Use	2.623	.677	4.569	6.977	.008
POST-survey measures
Knowledge of P-NLPRs after course	4.053	3.097	5.009	69.009	<.001
P-NLPRs Perception (POST)	5.462	4.349	6.575	92.518	<.001
P-NLPRs Familiarity (POST)	4.407	3.427	5.388	77.576	<.001

III.1.3.a.Perception of P-NLPRs usefulness

Within the first round of regression analyses, only “training in the use of mobile devices” was found to be significantly related to perceived usefulness (B= 3.2, p= .027) for the group of variables related to subject characteristics (i.e. demographics and years of experience) and working environment (i.e. type of institution and its attitude towards the use of mobile devices [MD] in the classroom). As for the PRE-survey and POST-survey assessment measures (knowledge, perception, familiarity and frequency of use), almost all individually showed a significant association with perceived usefulness of P-NLPRs (Table 5). Self-reported knowledge and experience with P-NLPRs were the only variables with no significant relationship with this scale.

Table 6. Univariate ordinal regression analyses for intention to use P-NLPRs after the course. Bold text for significant results.

	B	CI 95%		Wald	P
Subject characteristics and working environment
Gender	1.254	-.517	3.024	1.927	.165
Age	.634	-.340	1.608	1.627	.202
Years of Experience	-.82	-.388	.223	0.281	.223
Working Institution
Secondary School	-.889	-2.396	.618	1.336	0.248
Higher Education	-.992	-2.535	.551	1.586	0.208
Adult Education	-.779	-2.286	.729	1.025	0.311
Other (Ref)	0	-2.396	.618	1.336	0.248
Institution fosters MD	.905	.039	1.771	4.195	.041
MD training	.941	-0.56	1.937	3.424	.064
P-NLPR Training	.100	-1.138	1.337	0.025	.874
PRE-survey measures
Previous knowledge about P-NLPRs	.354	-.057	.764	2.855	.091
Previous experience with P-NLPRs	.307	-.051	.666	2.826	.093
P-NLPR Perception (PRE)	.904	.315	1.493	9.053	.003
P-NLPR Familiarity (PRE)	.953	.379	1.526	10.606	.001
P-NLPR Use	1.316	.480	2.152	9.515	.002
POST-survey measures
Knowledge of P-NLPRs after course	1.441	.897	1.984	27.010	<.001
P-NLPR Perception (POST)	4.547	3.106	5.988	38.250	<.001
P-NLPR Familiarity (POST)	1.286	0.748	1.823	21.988	<.001
P-NLPR Perceived usefulness	2.433	1.636	3.230	35.833	<.001

In a subsequent step, variables showing an independent significant association in the regression analyses were introduced in three different multiple regression models defined by group of origin (Subject characteristics and working environment, PRE-survey, and POST-survey measures) in order to select those most representative for each group. Only POST-survey familiarity (B= 2.64; p= <.001) and POST-survey perception of P-NLPRs (B= 3.66; p= <.001) remained significant in their association to perceived usefulness of P-NLPRs.

III.1.3.b. Intention to use P-NLPRs in the future

Results from the univariate ordinal regression analyses showed that, among those variables related to subject characteristics and working environment, only working in institutions encouraging the use of mobile devices had a significant positive association with the intention to use P-NLPRs (B= .905; p= .041). Other variables such as gender, age, or years of experience did not show significant association with a positive intention. Similarly, having received training in the use of MDs or P-NLPRs, or previous knowledge and experience using P-NLPRs were not significantly related to the “intention outcome” (Table 6).

Both PRE-survey and POST-survey measures on perception and familiarity, as well as PRE-survey use of P-NLPRs and POST-survey knowledge and perceived usefulness of P-NLPRs after completing the module were significantly associated with the intention to use P-NLPRs in the future (Table 6). Again, significant variables were introduced stepwise in a final multivariate model following the steps described above. The results show that only one of those variables remained in significant association with the future intention to use P-NLPRs (Figure 3): POST-survey perception of P-NLPRs (B= 4.494; p= <.001). All other variables lost significance when entering the model in subsequent steps.

III.1.3.c. Changes after the course and main outcomes

As a further check of the course effect, an analysis was performed on whether those changes that had been observed in knowledge, perception, and familiarity with P-NLPRs were related to the main outcomes of perceived usefulness and intention to use P-NLPRs. Hence, change scores (i.e., POST minus PRE score) of knowledge, perception and familiarity were used as predictors in a linear regression model using perceived usefulness as outcome. Both perception change (B=1.20; CI 95% .032, 2.364; Wald=4.05; p=.032) remained significantly associated with perceived usefulness in the multivariate model.

The same analytical procedure with the intention to use P-NLPRs as outcome yielded similar results but only for change in perception of P-NLPRs, which showed a significant association with an intentional attitude (B=1.22; CI 95% .530, 1.914; Wald=11.98; p=.001) in the multivariate analysis, even when adjusting for change in knowledge and familiarity.

III.1.4. Interviews

The interviews took place five years after the completion of the module (see 3.4), which provided the researchers with an unusually long post-treatment evaluation of the impact of the training on language teachers’ careers. The interviewees declared that completing the module had indeed been useful for them and that they had been making use of some of the P-NLPRs. Table 7 offers an overview of which tools they have used in their teaching practice.

In spite of generally showing a positive attitude towards the tools, the participants report having actually used highly familiar resources such as online dictionaries or text-to-speech technologies. There seems to be three main reasons behind their reluctance towards using some of the remaining resources: a) user-friendliness and visual appeal of the tools; b) appropriateness for the teaching context; and c) appropriateness for the learner context, such as the perception that certain resources belong to an academic environment. Additionally, subject C reports the impossibility of using online tools that require any kind of registration for her underage students.

All the subjects, however, agree on the importance of using authentic language in language education. They report designing their teaching materials with authentic pieces of language in mind.

“Years ago I taught A1 […] and I was always against slowing down audio clips. […] You have to listen to it at natural speed, because if I talk to you very slowly […] and then you go out and listen to someone who speaks slightly faster, you will not understand anything” (Subject A)

The subjects show some reservations about Data-Driven Learning and its actual in-class viability. The participants believe that their learners lack the necessary interest to approach language learning with such a mindset. More specifically, they remarked that raising language awareness may not result in the desired outcomes, as learners usually study foreign languages for practical reasons (speaking, reading, writing…) rather than for the sake of knowledge. ‘I believe that most people have no interest in knowing how language works’ (Subject B). They, however, find DDL-related P-NLPRs highly interesting for learners with a personal interest in expanding their linguistic knowledge, though, such as undergraduates or postgraduates in modern languages.

The interviewees agree on individualized learning improving teaching practice and reflection, yet all of them cite difficulties in its implementation due to highly populated classrooms and lack of effective time to carry out truly personalized learning. Subject C says: ‘I have 25 [students] per class. It is difficult [to implement] individualization. Besides, the curriculum is common to all of them.’

Furthermore:

“If we are aware that we have students that learn better in a visual way, then we will try to use visual-heavy resources in order to better get to them as well, right? If we have other students with a more developed linguistic intelligence, hey, let’s […] combine different kinds of activities so we can reach every single one of them. I wish we could make individualized content for each one of our students, but the truth is […] we have in class more than a hundred or so students.” (Subject D)

Table 7. Overview of the frequency of use of P-NLPRs after the course

P-NLPR	A	B	C	D
Text-to-speech technologies	Yes, especially when there is a specific word, she wants students to hear	Yes, especially to help students with their presentations	Yes, she used them in class and the students appreciated it, especially at lower levels	Yes, he uses them sometimes
Online dictionaries	Yes, constantly	Yes, constantly	Yes	Yes, frequently
Visual Representation of word clusters	Yes, sometimes as an activity in class	No, but she teaches her pre-service learners how to use it	She uses semantic maps, but she doesn’t use the tool	No, although he thinks depending on how it is implemented it could be viable
Corpora & Specialized corpora	No, only the instances shown in online dictionaries	No, she believes it can be useful for English Studies undergraduates	No, she thinks it is not user-friendly enough for her students	No, he does not think it would be useful in class
Social Networking Services	No, she doesn’t like them	No, she doesn’t like them	No. There are legal concerns at her institutions	No. However, he thinks they are useful as students are quite familiarized with them. Its success would also depend on the teacher’s own familiarity
Word lists & Frequency counts	No, she finds it interesting for an academic environment	Not for class, but to inform her own research	No, but she finds them interesting	No.
Vocabulary Profiling Tools	Yes, especially to inform assessment design	No, she thinks her students would lose interest	No, she does not think it would be useful for her secondary education students	No, he thinks these tools can help inform teacher materials, but they are more useful for an academic context
Automated POS taggers	No, she finds it interesting for an academic environment	No, she thinks its use would be more appropriate for English Studies undergraduates	No, it doesn’t fit with her institution’s philosophy	No. It’s too complicated for his students
Lemmatizers	No, the tool seems too academic to use in class	No, but she may have	No	No. He thinks they could be occasionally helpful
Text summarization tools	No, but she finds them interesting	She teaches it to her pre-service learners	No, but seems open to use it to inform her teaching materials	No, although he knew about it he never used it in class
Resource-linked text builders	Not really	No	She thinks it’s useful; she reckons doing similar work by herself	No, but he thinks it could be useful

The participants note that the tools could help in the process of individualization, especially in relation to informing material design:

‘Yes, I think they do [help with individualization]. […] For example, with every new edition of the textbook they [the publishers] improve the online platform for the students. […] It includes many different resources’ (Subject A).

‘I believe they can [help], a lot. For instance, [automatic summarizers], the ability to adapt to different levels. I think that may help me greatly to personalize my teaching.’ (Subject B).

‘Honestly, I would need to try it, right? At first sight, some of them seem viable. They could help me in preparing materials’ (Subject C).

‘Of course they [the P-NLPRs] can help […], by creating different types of activities, and we know there are some students who, due to their individual characteristics, will benefit from this’ (Subject D).

III.2. Discussion

This study offers empirical information about language educators’ perception towards adopting P-NLPRs for their language teaching. The data collected confirms that the training module designed for this study targets intention and perceived usefulness successfully; on the other hand, factors such as institution support has been suggested as being influential on subjects’ intention to use P-NLPRs. Research on DDL has been mostly limited to tertiary education members, so the perspectives this study provides from in-service educators covering a diverse range of teaching institutions and levels (i.e. secondary education) contribute to underscore some of the issues that need addressing in future research. These results suggest a division within the different P-NLPRs into three main groups: a) tools to help learners learn; b) tools to help teachers teach, and c) tools to help expand linguistic knowledge, provided the individual necessary motivation for it. The following sections will connect the results collected with the RQs outlined at the end of section 2.

III.2.1. Has the TELL-OP training module in the use of P-NLPRs succeeded in contributing to the promotion of language teachers´ perceived usefulness of P-NLPRs?

The initial answer seems positive, as the POST-survey participants said they may use P-NLPRs in their teaching in the future (Table 4). The only exception to this positive pattern was the P-NLPRs perception scale. In this case, the module managed to increase how participants perceive P-NLPRs, albeit non-significantly. This result could be explained by the high expectations about P-NLPRs the subjects declared prior to participating in the course, illustrated by an already high score at PRE-survey. This interpretation concurs with the literature, which has noted a generalized positive attitude among educators towards P-NLPRs (Pérez-Paredes et al., 2018).

In particular, P-NLPR perception at POST-survey remains the most important variable associated with perceived usefulness and intention to use P-NLPRs. This association is robust enough to maintain its significance, even after control for other variables takes effect. Even though the change in the perception score between the PRE- and POST-survey was slight, its relation to both outcomes is still significant. The measure of perception of P-NLPRs seems to summarize all other variables and encompass the main factors influencing intention to use P-NLPRs in the future. Familiarity at POST-survey is the other scale (see Table 2) that appears to be independently associated with perceived usefulness, although to a lesser extent. While significant association does not imply causality, results suggest that both encouraging familiarity with P-NLPRs and increasing teachers’ perception of P-NLPRs may play an important role in increasing the intention to use P-NLPRs for language teaching. In fact, participants declared that increasing knowledge about P-NLPRs strengthened intentions to use P-NLPRs in the future, in line with the literature (Littlejohn & Hood, 2017). Further exploration would thus be necessary to assess the extent to which these factors are essential for the potential adoption of P-NLPRs by the education community, perhaps by considering the broader scope of teacher knowledge covered in T-PACK (Mishra & Koehler, 2006).

Participating in the training activity seems to have contributed to increasing perceived usefulness of P-NLPRs, as it was strongly associated with positive changes in perceptions about and familiarity with P-NLPRs after the course. In fact, perceived usefulness shows one of the strongest associations with intentions in the univariate analysis. These results would support the literature suggesting perceived usefulness to be an important factor in increasing intention (Kelly, 2014; Lai, 2013; Lai et al., 2014), which strongly influences whether users will ultimately adopt the tools (Davis, 1989; Venkatesh et al., 2003). Moreover, these outcomes seem to imply that focusing on how P-NLPRs can be useful for learners should be important to consider when designing a course, as suggested by Lai et al. (2014). This finding is potentially relevant for future teacher training.

Institutions play a pivotal role in teacher development, the data suggests. Working in an institution that encourages the use of mobile devices (MDs) for language teaching appears to show a strong association to familiarity and having received training in the use of MDs with intention to use. The literature has stressed repeatedly the importance of providing sustained, supported learning opportunities grounded in immediate teaching contexts that show innovative ways to improve teaching practices (Knobel and Kalman, 2016). Even though none of the PRE-survey measures outscore the strength of the association of POST-survey measures (Tables 5 and 6), the latter are consistently, in statistical terms, more robust. Additionally, P-NLPR perception at POST-survey maintains its association to perceived usefulness and intention after controlling for knowledge, familiarity or even actual frequency of use of P-NLPRs at PRE-survey. This suggests that the impact of the course may go beyond the previous experience of the participants with P-NLPRs.

III.2.2. After five years of hands-on experience, how do in-service teachers perceive the viability of using P-NLPRs in their respective educative contexts?

The interviewees show positive attitudes towards the P-NLPRs in general, yet not all the presented P-NLPRs were appraised equally. Taking into account the participants’ declared delayed usage of the individual tools (table 7) and their perspectives on the potentiality of each tool for the classroom, we have classified the P-NLPRs into three distinct groups: tools that help learners learn, that is, aimed at helping learners perform their activities in-class (i.e. Text-to-speech technologies, online dictionaries), tools that help teachers teach and inform their material design (i.e. automatic text summarizers, visual representation of word cluster tools, lexical analysers), and tools to expand linguistic knowledge (i.e. lemmatizers, POS taggers, corpora), suitable for learners who are interested in developing their language awareness.

The time-consuming process of selecting, designing, and implementing DDL-based materials, teachers’ perceived inability to interpret their students’ findings, and the lack of available, ready-to-use DDL-informed materials accessible to teachers and students have been identified as hurdles to the implementation of language data-related activities in the classroom in agreement with Chambers, (2019), Poole, (2020), Zareva, (2017). This is reflected on the subjects’ perspective on the more ‘DDL-like’ tools (i.e. corpus-related tools). The very notion of raising language awareness in learners that belong to a non-linguistics related background seems to be perceived as futile. They cite student predisposition towards learning languages, as the average learner usually takes up a foreign language mainly for practical reasons, that is, to be able to competently communicate with it. Thus, the difficulty of introducing corpora as a recursive resource in L2 teaching practices (Latif, 2021) suggest that P-NLPRs could help redirect DDL research towards reaching a ‘feasibility scenario’ in which DDL-based materials and textual data are tailored to the learners ’ needs (Pérez-Paredes, 2010).

Additionally, there is a general reluctance towards using social media as language learning tools. Only one of the participants (subject B) seemed open to the idea, due to his undergraduate students’ familiarity with social media apps, which would ease the implementation of activities based on MALL (Mobile-Assisted Language Learning) and CALL media. It is, however, a contentious topic for both educators who dislike and/or are unfamiliar with how these applications work and for teachers in a secondary education environment. The latter case seems to raise concerns due to learners being underage and the lack of control over the apps’ usage.

III.2.3. According to in-service L2 teachers, how can NLP tools favour personalized and individualized learning across the board?

The participants found that the majority of the P-NLPRs were more useful to inform their homemade teaching materials rather than to let the students use them by themselves. They particularly pointed at automatic text summarization tools, lexical profiling resources or resource-linked text builders as potentially helpful resources, as they allow for the adaptation of teaching materials to the specific needs of the learner, whether by highlighting linguistic issues that need addressing or by calibrating the linguistic level of a particular teaching material.

The idea of individualized learning seems attractive to all the participants, although the level of viability relies heavily on constraints from lack of available preparation time, curriculum flexibility and/or institution support. This becomes a common gripe amongst the participants, who feel they cannot implement innovative practices as freely as they would like due to limitations imposed by the curriculum and by the logistics of managing a large number of students per class. This relates to the literature calling for an increased effort from both curricula and institutions to adapt their policies to provide room for educators to implement alternate teaching practices (Chambers, 2019; Pérez-Paredes et al., 2018).

IV. CONCLUSIONS

This study conceptualized P-NLPRs as free-to-use digital tools that offer language-related information with the intent of informing the design and implementation of an individualized learning process. Such tools could also be used as vehicles for DDL-like activities in the classroom (Cantos, 2002; Granger et al., 2007; Pérez-Paredes et al., 2018, 2019), as they allow for the exploration of authentic linguistic data without the need to use a corpus. Training language teachers in the use of P-NLPRs in the classroom contributes to the development of new literacies, which have become an essential part of the skillset of educators (Kern, 2021; Trust et al., 2022).

Our results showed that the module significantly increased the knowledge about and the familiarity with P-NLPRs, which appear to be necessary —but not sufficient— factors for perceived usefulness and intention to occur. Knowledge, familiarity, and perception were related to perceived usefulness, and all four with intention to use P-NLPRs. Working in institutions encouraging the use of mobile devices may also have an indirect effect on intention, which stresses the relevance of the efforts from institutions to provide learning opportunities for teachers (Knobel & Kalman, 2016). Our interview data revealed that training efforts may need to go beyond knowledge-driven approaches and increase their focus on aspects such as usability of P-NLPRs, their role on students’ attraction and participation, and ease of integration in daily practice in order to effectively promote teacher adoption of innovative technology. This research contributes to filling some gaps in CALL and teacher training. Particularly, this research explored in-service language teachers with varying degrees of teaching experience, as opposed to other research designs (Taghizadeh & Basirat, 2022) that examine pre-teachers’ attitudes and perspectives in a one-site research design. The variety of countries where our teachers develop their practice offers a multi-site, wide-ranging perspective on the use of P-NLPRs, particularly across Europe, that is missing in the literature.

Some limitations should be considered when interpreting the results of this study, thus readers’ discretion is advised. The tools selected for this study are but a handful in the vast and ever-evolving landscape of P-NLPR design and development. The TELL-OP website (www.tellop.eu) includes a wide range of these tools in different languages. Similar tools as those covered in this study may be perceived differently due to differences in how they perform the language processing and how they present the data obtained, whether explicitly or implicitly —i.e. included as part of a larger learning platform (Katinskaia et al., 2018). The utilisation of P-NLPRs in more ‘subtle’ ways (i.e. without explicitly stating that the main purpose of using certain P-NLPRs is to raise language awareness) might result in a better engagement from teachers and students alike. Furthermore, despite the seemingly positive impact of the intervention, it must be noted that our outcomes are based on perceptions and self-reports and are by no means indicative of an actual increase in frequency of use of P-NLPRs. Moreover, sample size for the quantitative data is on the limit for a reasonable multivariate analysis and may be slightly underpowered for more complex analyses, including the exploration of inter-country differences, which could provide interesting insight in other areas, such as multilingualism-driven research.

Data collection and sampling procedures for the interviews and questionnaires were completed online, which could as well raise concerns about the trustworthiness of the responses. A larger participant pool would of course have provided additional power for greater accuracy and precision. It should be noted that almost half of our subjects declared having less than five years of experience in teaching when the survey was completed; this could likely point at a bias of self-selection, as their perceptions might be affected by the novelty of the resources rather than by actual hands-on practice. Further research should take this into account to explore the perspectives of experienced teachers, which might provide different insight on the matter.

Further research on the topic of teacher engagement with P-NLPR tools is necessary. There are several possible reasons behind instructors’ reluctance to include these resources that have not been covered in this paper, such as questions of quality of the tools, accuracy of the data processed or user-friendliness (Burstein, 2009; Burstein et al., 2012), to name a few. Further attempts at designing similar training modules should improve the sophistication of the tasks included and the depth of the information provided on the different types of P-NLPRs and their potential for language instruction. Such factors need be explored to advance on the outcomes of this research. A qualitative perspective may be needed to obtain more in-depth information about how teachers introduce P-NLPRs into their own conception and approach to language pedagogy. To this regard, the research presented here can be considered as a useful source of information as a starting point to design and guide the development of further inquiries.

The findings of our research suggest that integrating language teachers’ views on the uses of P-NLPRs can indeed increase the opportunities for further engagement with textual data and language awareness classroom activities. Text-driven examination of language in the L2 classroom does not have to be limited to corpus use, but rather expand their toolset to adapt to a much wider audience through the adoption of P-NLPRs. This perspective would concur with the literature stressing the importance of a more generalized adoption of DDL and DDL-like practices in the classroom (Chambers, 2019).

REFERENCES

Abdel-Latif, M. (2021). Corpus literacy instruction in language teacher education: Investigating Arab EFL student teachers’ immediate beliefs and long-term practices. ReCALL, 33(1), 34–48. https://doi.org/10.1017/S0958344020000129

Azungah, T. (2018), “Qualitative research: deductive and inductive approaches to data analysis”, Qualitative Research Journal, Vol. 18 No. 4, pp. 383-400. https://doi.org/10.1108/QRJ-D-18-00035

Bird, S., Klein, E. and Loper, E. (2009). Natural language processing with Python. Sebastopol: O’Reilly Media.

Boulton, A., & Vyatkina, N. (2021). Thirty years of data-driven learning: Taking stock and charting new directions over time. Language Learning & Technology, 25(3), 66-89.

Burstein, J. (2009). Opportunities for natural language processing in education. In A. Gelbulkh (Ed.), Lecture notes in computer science: Vol.5449. Computational linguistics and intelligent text processing. (pp. 6–27). Berlin, Germany: Springer-Verlag.

Burstein, J., Shore, J., Sabatini, J. Moulder, B. Holtzman,S., & Pedersen, T. (2012). The Language MuseSM system: Linguistically focused instructional authoring. ETS Research Report Series, 2012 (2), i-36.

Cabero, J., & Barroso, J. (2016). ICT teacher training: a view of the TPACK model / Formación del profesorado en TIC: una visión del modelo TPACK. Culture and Education, 28(3), 633-663. https://doi.org/10.1080/11356405.2016.1203526

Cantos Gómez, P. (2002). Integrating corpus-based resources and natural language processing tools into CALL. International Journal of English Studies, 2(1), 129-166.

Chambers, A. (2019). Towards the corpus revolution? Bridging the research–practice gap. Language Teaching, 52(4), 460-475. doi:10.1017/S0261444819000089

Chapelle, C. (2006). Autonomy meets individualization in CALL. Mélanges Crapel, 28.

Cobb, T. (1997). Is there any measurable learning from hands-on concordancing? System, 25, 301–315.

Cohen, J., Cohen, P., West, S.G., Aiken, L.S. (2003) Applied Multiple Regression / Correlation Analysis for Behavioral Sciences. Lawrence Erlbaum, London.

Criollo-C, S., González-Rodríguez, M., Guerrero-Arias, A., Urquiza-Aguiar, L. F., & Luján-Mora, S. (2024). A Review of Emerging Technologies and Their Acceptance in Higher Education. Education Sciences, 14(1).

Davis, F.D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of lnformation Technology. Management Information Systems Quarterly, 13(3), 319–339.

Elola, I., & Oskoz, A. (2017). Writing with 21st century social tools in the L2 classroom: New literacies, genres, and writing practices. Journal of Second Language Writing, 36, 52-60. https://doi.org/https://doi.org/10.1016/j.jslw.2017.04.002

Farr, F., & Hagen Karlsen, P. (2022). DDL pedagogy, participants, and perspectives. In R. R. Jablonkai & E. Csomay (Eds.), The Routledge Handbook of Corpora and English Language Teaching and Learning (pp. 329-343). Routledge.

Farrow, R., Pitt, R., de los Arcos, B., Perryman, L., Weller, M., & McAndrew, P. (2015). Impact of OER use on teaching and learning: Data from OER Research Hub (2013–2014). British Journal of Educational Technology, 46, 972–976.

Golonka, E., Bowles, A., Frank, V., Richardson, D., & Freynik, S. (2012). Technologies for foreign language learning: a review of technology types and their effectiveness. Computer Assisted Language Learning, 27(1), 70-105.

Granger, S., Kraif, O., Ponton, C., Antoniadis, G., & Zampa, V. (2007). Integrating learner corpora and natural language processing: A crucial step towards reconciling technological sophistication and pedagogical effectiveness. ReCALL, 19(3), 252-268. https://doi.org/10.1017/S0958344007000237

Huang, H., & Liou, H. (2007). Vocabulary Learning in an Automated Graded Reading Program. Language Learning & Technology, 11(3), 64-82.

Hsu, Y.-Y., & Lin, C.-H. (2020). Evaluating the effectiveness of a preservice teacher technology training module incorporating SQD strategies. International Journal of Educational Technology in Higher Education, 17(1). doi:10.1186/s41239-020-00205-2

Johns, T. (1991). Should You Be Persuaded: Two Examples of Data-driven Learning. In T. Johns & P. King (Eds.), Classroom concordancing (pp. 1-16). Centre of english language studies, The University of Birmingham.

Katinskaia, A., Nouri, J., & Yangarber, R. (2018). Revita: a Language-learning Platform at the Intersection of ITS and CALL. in C Nicoletta (ed.), Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), pp. 4084-4093, International Conference on Language Resources and Evaluation , Miyazaki, Japan, 07/05/2018 . < http://www.lrec-conf.org/proceedings/lrec2018/summaries/841.html >

Kelly, H. (2014). A path analysis of educator perceptions of open educational resources using the technology acceptance model. The International Review of Research in Open and Distributed Learning, 15(2), 26-42.

Kern, R. (2021). Twenty-five years of digital literacies in CALL. Language Learning & Technology, 25(3), 132–150. http://hdl.handle.net/10125/73453g

Kessler, G. Technology and the future of language teaching. Foreign Language Annals. 2018; 51, 205– 218. https://doi.org/10.1111/flan.12318

Knobel, M., & Kalman, J. (2016). Teacher learning, digital technologies and new literacies. In M. Knobel & J. Kalman (Eds.), New literacies and teacher learning: Professional development and the digital turn, (pp. 1-20). New York, NY: Peter Lang.

Lai, C. (2013). A framework of developing self-directed technology use for language learning. Language Learning & Technology, 17(2), 100–122.

Lai, C., Shum, M., & Tian, Y. (2014). Enhancing learners’ self-directed use of technology for language learning: the effectiveness of an online training platform. Computer Assisted Language Learning, 29(1), 40-60.

Latif, M. M. A. (2021). Corpus literacy instruction in language teacher education: Investigating Arab EFL student teachers’ immediate beliefs and long-term practices. ReCALL, 33(1), 34-48.

Le Foll, E. (2024). ‘Opening up’ Corpus Linguistics: An Open Education Approach to Developing Corpus Literacy among Pre-service Language Teachers. Second Language Teacher Education, 2(2), 161-186. https://doi.org/10.1558/slte.25371

Leńko-Szymańska, A. (2017). Training teachers in data-driven learning: Tackling the challenge. Language Learning & Technology, 21(3), 217–241.

Littlejohn, A., & Hood, N. (2017). How educators build knowledge and expand their practice: The case of open education resources. British Journal of Educational Technology, 48(2), 499–510.

Lotherington, H., & Ronda, N. (2014). 2B or not 2B? From pencil to multimodal programming: New frontiers in communicative competencies. In J. Guikema & L. Williams (Eds.), Digital literacies in foreign and second language education (pp. 9–28). CALICO.

Mekheimer, M. (2012). Assessing Aptitude and Attitude Development upon Teaching Translation Skills using LMS and an Online Dictionary. CALICO Journal, 29 (2), 321-340.

Meunier, F. (2020). A case for constructive alignment in DDL: Rethinking outcomes, practices and assessment in (data-driven) language learning. In P. Crosthwaite (Ed.), Data-Driven Learning for the Next Generation: Corpora and DDL for Pre-tertiary Learners (pp. 13-31). Routledge.

Meunier, F. (2022). Revamping DDL: affordances of digital technology. In R. R. Jablonkai & E. Csomay (Eds.), The Routledge Handbook of Corpora and English Language Teaching and Learning (pp. 344-360 ). Routledge.

Mishra P., Koehler J. (2006). Technological pedagogical content knowledge: A new framework for teacher knowledge. Teachers College Record, 108, 1017–1054.

Nilsson, N. (2009). Natural language processing. In N. Nilsson (Ed.) The quest for artificial intelligence (pp. 103–113). Cambridge: Cambridge University Press.

Ordoñana, C., Pérez-Paredes, P., Aguado-Jiménez, P. (2018, July). Language Teachers’ Perceptions on the use of P-NLPRs. Paper presented in CALL your Data 2018, Bruges, Belgium. Retrieved from https://www.call2018.org/wp-content/uploads/2018/07/proceedings-CALL-2018.pdf

Owen, C. (2017). Surveys. In J. Swain (Ed.), Designing research in education (pp. 123–147). London: Sage.

Pedhazur, E.J. (1997) Multiple regression in Behavioral Research. Harcourt Brace, Fort Worth.

Pérez-Paredes, P. (2010). Corpus linguistics and language education in perspective:  Appropriation and the possibilities scenario. In T. Harris & M. Moreno Jaén (Eds.), Corpus linguistics in language teaching (pp. 53–73). Peter Lang.

Pérez-Paredes, P. (2022). A systematic review of the uses and spread of corpora and data-driven learning in CALL research during 2011-2015. Computer Assisted Language Learning, 1-26. https://doi.org/10.1080/09588221.2019.1667832

Pérez-Paredes, P., Ordoñana, C., & Aguado-Jiménez, P. (2018). Language teachers’ perceptions on the use of OER language processing technologies in MALL. Computer Assisted Language Learning, 31:5-6, 522-545

Pérez-Paredes, P., Ordoñana, C., Van de Vyver, J., Meurice, A., Aguado-Jiménez, P., Conole, G., & Sánchez, P. (2019). Mobile data-driven language learning: Affordances and learners’ perception. System, 84, 145-159.

Poole, R. (2020). “Corpus can be tricky”: revisiting teacher attitudes towards corpus-aided language learning and teaching. Computer Assisted Language Learning, 1-22.

Romeo, K., & Hubbard, P. (2010). Pervasive CALL learner training for improving listening proficiency. In M. Levy, F. Blin, C. Siskin, & O. Takeuchi (Eds.), World CALL: International perspectives on computer-assisted language learning. New York: Routledge

Stockwell, G. (2022). Theory in MALL. In Mobile Assisted Language Learning: Concepts, Contexts and Challenges (pp. 90–113). chapter, Cambridge: Cambridge University Press.

Strauss, A., & Corbin, J. (1994). Grounded theory methodology: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 273–285). Sage Publications, Inc.

Taghizadeh, M., & Basirat, M. (2022). Investigating pre-service EFL teachers’ attitudes and challenges of online teaching. Computer Assisted Language Learning, 1-38.

Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage Publications, Inc.

TELL-OP. (2015). Our Aims. Retrieved from http://www.tellop.eu/our-aims/

Trust, T., Maloy, R. W., & Edwards, S. (2022). College student engagement in OER design projects: Impacts on attitudes, motivation, and learning. Active Learning in Higher Education. https://doi.org/10.1177/14697874221081454

Vai, M., & Sosulski, K. (2015). Essentials of online course design: a standards-based guide (2nd ed.). Routledge.

Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science, 46(2), 186–204. doi: 10.1287/mnsc.46.2.186.11926

Venkatesh, V., Morris, M. G., Davis, F. D., & Davis, G. B. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

Zareva, A. (2017). Incorporating corpus literacy skills into TESOL teacher training. ELT Journal, 71 (1), 69-79.

Zhu, Q. (2023). Natural Language Processing in Teacher Training: a systematic review. Lecture Notes in Education Psychology and Public Media, 18(1), 83-90. https://doi.org/10.54254/2753-7048/18/20231293

Zhu, D. H., & Chang, Y. P. (2015). Effects of interactions and product information on initial purchase intention in product placement in social games: the moderating role of product familiarity. Journal of Electronic Commerce Research, 16(1), 22-33.

Received: 11 November 2024

Accepted: 18 December 2024