An Analytical Study on Student Perceptions of Using ChatGPT in Language Translation

Zakaryia Almahasees

z_almhasees@asu.edu.jo

Applied Science Private University, Jordan

Hussein Abu-Rayyash

haburayy@kent.edu

Department of Modern & Classical Language Studies,

Kent State University, USA

Sameer Naser Olimat

solimat@hu.edu.jo

Department of English Language and Literature, Faculty of Arts

The Hashemite University, Jordan

Dana Mahadin

d.mahadin@bau.edu.jo

English Language Department, Salt Faculty of Human Sciences

Al-Balqa Applied University, Jordan

ABSTRACT

This study explores user perceptions and practical employment of ChatGPT Translation among students involved in English-Arabic translation tasks. For the quantitative part method, data were collected from a diverse assembly of 100 participants, comprising demographic queries and a constructed 20-item Likert scale questionnaire. The collected data were categorized and critically examined under five principal constructs: Efficiency, Accuracy, Ease of Use, Trustworthiness, and Overall Preference. The findings indicate a predominantly positive perception of ChatGPT among the participants. In addition, detailed analysis reveals novel insights, such as the considerable appreciation for ChatGPT’s role in enhancing translation efficiency and the high level of trust expressed in its ability to maintain the confidentiality of translated work. Another significant finding is the tool’s competitive edge, with participants favouring ChatGPT over other translation tools. Furthermore, the findings underscore the importance of extending the research landscape on AI-assisted translation tools. Translation platforms foster their effective integration into the industry and understanding of their potential impact on the future of translation pedagogy.

Keywords: ChatGPT; English-Arabic; User perceptions; AI-assisted translation tools; Translation pedagogy

I. INTRODUCTION

The interplay between technology and the translation process has significantly impacted global modes of communication. Historically, translation was a demanding and cost-intensive process, predominantly the purview of governments and large corporate entities. However, with the advent of innovative technological advancements in the present era, accessibility to translation services has substantially broadened, revolutionizing the global communication paradigm (Almahasees, 2021).

Technological interventions are implemented in many ways to augment the translation process. For example, machine translation (MT) capitalizes on the synergies between translation and technology. Current neural MT systems, as a specialized form of computer-assisted translation systems, employ intricate algorithms to transmute text from the source language (SL) into the target language (TL), signifying the capabilities of artificial intelligence (AI) (Almahasees , 2017; Hasyim et al., 2021).

Translation Memory is one of the prominent technological tools used to improve translation results. It is a segmented, translated text database that translators can reuse for future work. This significantly improves consistency in a much lesser workload. While Translation Memory systems are part of CAT tools, they differ fundamentally from MT. TM is based on human-translated segments stored in its database, while MT, especially AI-driven MT, generates the translation automatically through algorithms and neural networks. Both tools can be integrated into translation workflows, complementing each other to achieve efficiency and quality on both levels (Cai et al., 2021; Al-Taher, 2019). Besides, one of the major technological interventions, post-editing (PE) can be utilized to enhance translation quality. Generally, PE is a process in which a human translator carefully reviews and edits machine-generated text to make it more accurate, fluent, and readable. This is often dovetailed with MT systems since it represents an effective method to improve translation quality (Li & Chen, 2019). Moreover, PE holds great potential for translators’ training (Krings, 2001).

Integrating technology with the translation process enables users to translate texts quickly, efficiently, and accurately. Consequently, individuals worldwide can engage in seamless cross-cultural exchanges, substantially altering the communication landscape. The advent of instant translation underscores the innumerable benefits inherent in the symbiosis between translation and technology (Almahasees, 2018). This research investigates the perception and attitude of translation students concerning the use of ChatGPT, an AI-based tool, and conventional MT when performing their English<>Arabic translations. This research does not aim directly at assessing the efficacy of these tools but rather seeks to understand how students perceive their effectiveness and potential. With that in mind, the students’ perception is valuable in terms of user experiences and attitudes but may not always reflect the real efficacy or accuracy of the tools. The study attempts to bring into focus how students interact with such technologies and the perceived impacts on the performance of translation tasks.

Technology translation tools have catalysed intercultural communication improvement. The integration of technology into translation has greatly improved the capability to transcend linguistic and cultural barriers, allowing more rapid, efficient, and increasingly accurate translation processes. These have stimulated mutual understanding and fostered global interconnectedness, making the world society even more integrated with each passing day, according to O’Hagan (2019).

When applied appropriately in the translation process, translation tools like Google Translate and Microsoft Translator offer manifold benefits. They aid in minimizing typographical and grammatical errors to ensure the quality of translated content. They also have the facility to archive previously translated material, which could be a resource for subsequent translation projects. Some of these tools automate translation management tasks, greatly reducing the time to market for delivering products to the translation market. Consequently, this enables companies to produce more content within a shorter period than would have been possible earlier. While acknowledging the current limitations of MT dealing with distant languages, its rising popularity is a testament to its potential. Advancements have been made and are continuously in the making in order to enhance the localization capability of MT, after which it will be able to adapt and translate text into various languages. Moreover, ease of usage makes MT a tool useful for professional translators and laypeople with ease (Almahasees, 2021). Given the broad integration of translation with technology and its resultant benefits, the current study tries to investigate the views of translation students on using ChatGPT in translation.

Developed by OpenAI, ChatGPT is a sophisticated chatbot at the forefront of AI-driven natural language processing (NLP) applications. Harnessing a broad array of NLP skills, including storytelling, logical reasoning, code debugging, and machine translation, ChatGPT showcases the sheer depth and breadth of AI’s potential. This suite of capabilities empowers ChatGPT to foster dynamic, human-like discussions by providing valuable assistance in numerous tasks, such as drafting emails and composing articles (Hariri, 2024).

ChatGPT, unveiled in November 2022, is underpinned by a monumental large-scale language model called Generative Pre-trained Transformer (GPT) pioneered by OpenAI. It represents an evolution of the foundational technologies encapsulated in GPT-3.5 and GPT-4. To fine-tune the performance of GPT-4, OpenAI applied the concept of transfer learning prompt. The result is a chatbot capable of managing extensive text volumes, accommodating over 25,000 words, and extending its translation capabilities across more than 100 languages (Haleem et al., 2022).

The formidable capabilities of ChatGPT highlight its potential to seamlessly synergize with other MT systems, contributing to the broader ecosystem of translation services (Jiao et al., 2023). Businesses and professionals can significantly enhance their translation efficiency, accuracy, and scope by incorporating such AI-driven tools in their workflow. The innovative strides made by ChatGPT thus illuminate the vast potential of AI in revolutionizing the translation industry (Peng et al., 2023).

MT is a cumbersome process in linguistic and computational terms and is based on a series of steps which are systematic and well-sequenced. The first step includes a very close examination and comprehension of the input SL, based on which an internal representation or mental model of the TL is constructed. This internalized representation is intended to capture the meaning, syntax, and style of the source text to provide an all-inclusive and accurate transposition into the TL. From this mental model, an appropriate form in the TL is then derived and manipulated to be structured according to the linguistic and cultural norms of the TL audience. The final step of this process is generating the translated output, which is the tangible product of these complex transformations (Almahasees, 2021).

MT, therefore, is a subfield of computational linguistics that focuses on translating texts from one language to another without human intervention. The MT paradigm is divided into four distinct yet overlapping areas: rule-based MT, statistical MT, hybrid MT, and neural MT. The advent of neural MT and the subsequent release of Google’s NMT system in 2016 precipitated a sudden surge in translation quality. Subsequently, there has been an astronomical increase in the availability of translation software incorporating NMT.

ChatGPT represents another paradigm for MT. As Jiao et al. (2023) note, GPT-4 utilizes large, advanced language models for various NLP tasks in many languages. It creates a unified MT paradigm that incorporates the strengths of several previous approaches into one coherent framework. Furthermore, ChatGPT employs translation databases that streamline and enhance the translation process and offer a more nuanced understanding of language pairs, which, in turn, contributes to producing superior translations (Jiao et al., 2023). It can be concluded that the rapid evolution of MT technologies, exemplified by GPT-4, promises to usher in new heights in translation accuracy, efficiency, and scope.

II. LITERATURE REVIEW

MT is, therefore, a subfield of computational linguistics and concentrates on the translation of texts from one language to another without human intervention. The MT paradigm comes in four clear-cut, yet overlapping areas: rule-based MT, statistical MT, hybrid MT, and neural MT. This is where Neural MT, starting this trend, followed by the release of Google’s NMT system in 2016, was responsible for the sudden spike in translation quality. Subsequently, there has been an astronomical increase in the availability of translation software incorporating NMT.

According to Jolley and Maimone (2022), MT plays a significant role in language learning and instruction. They indicated that most students rely heavily on MT to complete writing assignments, especially when searching for specific words or phrases in their language. Despite recognizing the limitations of MT, students value it as a time-efficient tool for enhancing their language skills (Monib et al., 2024). Interestingly, the usefulness of MT in the L2 classroom generates mixed opinions among teachers and students; some teachers resist students’ usage of MT, while other teachers perceive its pedagogical significance and advocate for further MT training for themselves and students (Almahasees, 2021).

Broadening the scope of MT application, Anazawa et al. (2013) have demonstrated its beneficial role in diverse academic disciplines, such as Nursing, Science, and Languages. For instance, MT systems have helped global nursing professionals comprehend scholarly articles. Furthermore, Biology and Microbiology students frequently resort to MT resources to concurrently study languages in their academic courses (Archila & de Mejía, 2020; Farghal & Haider, 2024). Apart from that, MT evaluation measures the quality of the translations and interpretations provided by second-language writers through grammatical accuracy analysis, semantic fidelity, lexical choice, and fluency. These measures are responsible for locating errors, making judgments about overall coherence, and determining to what degree the translated text actually conforms to the original meaning. However, using MT tools in digital collaborative writing could pose challenges for students, implying that using MT in English academic writing necessitates enhancing students’ MT literacy (Almahasees, 2018).

Sujarwo (2020) conducted an analytical study to elucidate the perceptions of using MT systems in teaching Translation courses at Megarezky University. By surveying 13 Translation students, the study revealed translation students at the English Education Department in the sixth and seventh semesters. Translation students often take several semesters to intensively revise and refine words, phrases, sentences, and paragraphs generated through MT-a process called post-editing. Such a reality only underlines the difficulty that students experience in developing the competencies for critical analysis and enhancement of the output produced by MT, which should remain at the centre of their training. Conversely, Lee (2020) investigated the impact of MT on students’ writing by implementing a writing test using MT. The study demonstrated that MT aided in reducing lexico-grammatical errors and enhancing revisions, thereby improving students’ revision skills and understanding of writing.

Building on the recent developments in MT, researchers have scrutinized the use of MT tools for academic purposes, particularly in foreign language instruction. Lee (2023) extensively analysed 87 reports published between 2000 and 2019 concerning MT use in FL education. The study showed increased relevant publications over the past years, coupled with improved MT quality. However, the study also unveiled students’ complex responses toward MT and highlighted the discrepancy between instructors’ and students’ perceptions of this tool.

Adding to the discourse on the potential of NMT in foreign language instruction, Klimova et al. (2023) utilized NMT to unearth best practices for language teaching. The meta-analysis results indicated that NMT effectively enhances communicative (speaking and writing) and receptive (reading and listening) language skills and mediation skills, which are integral to the translation process.

Deng and Yu (2022) presented a comprehensive literature review on MT-assisted language acquisition, focusing on potential users, users’ perceptions, and MT integration into teaching and learning. The research indicated that most MT users are college and university students. However, teachers and students held ambivalent views due to various reasons. It has been found that MT integration occurred in four stages: preface, demonstration, task assignment, and evaluation. Future improvements to the MT integration process may include the addition of new functionalities.

Focusing on the potentiality of ChatGPT as a translation engine, researchers evaluated its effectiveness in dealing with European and foreign languages (such as Chinese). The study found that ChatGPT performs comparably to commercial translation tools (such as Google Translate) in high-resource European languages, yet its effectiveness diminishes for low-resource or foreign languages. In a similar vein, other researchers aimed to enhance the models used by ChatGPT to improve translations across all languages. They concluded that the performance of ChatGPT could be improved by training the machine to handle various training sets, focusing on task information, introducing domain information to enhance the generalization ability of ChatGPT, and addressing its tendency to generate incorrect translations for non-English-centric MT tasks.

III. METHODOLOGY

This research adopts a quantitative approach, examining the responses of a group of 200 undergraduate and postgraduate translation students to a 3-point Likert Scale questionnaire created with Microsoft Forms, consisting of 20 close-ended statements. The questionnaire was divided into two main sections; the first collected demographic data, such as gender, age, frequency of use, and type of device used. In the second section, the respondents were asked to share their experiences with their use of ChatGPT for translation tasks. This section included 20 closed-ended statements classified into five distinct themes: (1) efficiency, (2) accuracy, (3) ease of use, (4) trustworthiness, and (5) overall preference. A snowball sampling method was utilized, encouraging participants to share the questionnaire link among fellow translation students. The total number of respondents was 220. Twenty of them served as pilots to evaluate the reliability of the questionnaire and were subsequently removed from the study sample.

The survey used in this study was reviewed by a four-member panel specializing in machine translation to obtain their insights and recommendations on the five constructs and 20 items. The feedback, which revolved around the questions of the questionnaire, was incorporated before the questionnaire’s finalization. Furthermore, to ensure the internal consistency of the sub-scales, a Cronbach’s alpha test was carried out on a subset of 7 participants, who were subsequently removed from the study, affirming the research instrument’s reliability (Cronbach, 1951). Table 1 presents the results for the questionnaire’s 20 items and their corresponding construct reliability.

Table 1. Reliability analysis through Cronbach’s alpha results (Cronbach, 1951)

Construct	No. of Items	Cronbach’s Alpha
Efficiency	4	0.82
Accuracy	4	0.91
Ease of Use	4	0.92
Trustworthiness	4	0.89
Overall Preference	4	0.83
All Variables	20	0.76

Table 1 showcases a considerable level of reliability, suggesting significant internal consistency. A reliability coefficient of 0.70 or more is regarded as “acceptable” in social science research (Nunnally, 1978). Correlation is a bivariate analytical method used to ascertain the direction and intensity of the link between two variables. The sign of the coefficient signifies the direction of the relationship, with “+” denoting a positive correlation and “–” a negative one. Regarding strength, the correlation coefficient’s value lies between +1 and -1. A coefficient closer to 0 suggests a weak link, while a value of +1 or -1 indicates a strong correlation. Considering these results, Pearson Correlation analyses were performed (see Table 2).

Table 2. Pearson Correlation Matrix for the Five Constructs

	Efficiency	Accuracy	Ease of Use	Trustworthiness	Overall Preference
Efficiency
Pearson Correlation	1	0.91*	-0.2	0.28	-0.60*
Sig. (2-tailed)		0.004	0.67	0.54	0.02
Accuracy
Pearson Correlation	0.91*	1	0.13	0.45**	-0.80**
Sig. (2-tailed)	0.004		0.79	0.009	0.003
Ease of Use
Pearson Correlation	-0.2	0.13	1	0.77*	-0.11
Sig. (2-tailed)	0.67	0.79		0.02	0.82
Trustworthiness
Pearson Correlation	0.28	0.45**	0.77*	1	-0.29
Sig. (2-tailed)	0.54	0.009	0.02		0.53
Overall Preference
Pearson Correlation	-0.60*	-0.80**	-0.11	-0.29	1
Sig. (2-tailed)	0.02	0.003	0.82	0.53
Note: p < 0.01, *p < 0.05.

Table 2 provides the Pearson Correlation Matrix for the five constructions. The results denote statistically significant relationships among various constructs, with some Pearson Correlation Coefficients surpassing 0.50. The most potent correlation was observed between ‘Efficiency’ and ‘Accuracy,’ with a Pearson correlation value of 0.91**, which is significant at the 0.01 level (2-tailed). Another significant relationship exists between trustworthiness and accuracy, with a Pearson correlation value of 0.45**, which is significant at the 0.01 level (2-tailed). Additionally, there was a significant negative relationship between ‘Efficiency’ and ‘Overall Preference,’ with a Pearson correlation value of -0.60*, significant at the 0.05 level (2-tailed).

IV. RESULTS

IV.1. Demographic Information

In the demographic section of the questionnaire, the researchers sought to accumulate a substantial amount of data concerning the participants’ gender, age brackets (17–24, 25–34, or 35+), frequency of interaction with ChatGPT (rarely, sometimes, often, or always), and choice of device for accessing the A.I. model (desktop, laptop, tablet, or smartphone).

As presented in Table 3, the participant cohort comprised 28 males, 27.5% of the sample, and 74 females, 72.5%. This disproportionate representation of females could be traced back to the overall gender dynamic in the translation field, where females have been found to have a dominant presence (Simon, 2003; Al- Salman & Haider, 2024). This skewness may offer unique gender-specific perspectives and insights into using AI translation tools like ChatGPT.

Considering the age demographic, the data shows that most of our undergraduate participants fall into the 17 to 24 age bracket (46.1%), followed by those who are postgraduate in the 25 to 34 range (38.2%). Graduate participants aged 35 and older account for the remaining 15.7%. The younger demographic’s prominence in this study could suggest a higher acceptance and engagement level with AI-driven tools like ChatGPT. This is unsurprising given the younger generation’s known proclivity towards digital technologies and the growing integration of such technologies in their academic and professional lives.

Table 3. Participants’ Demographic Information

Variable	Category	Counts	% Percentage
Gender	Male	28	27.50%
	Female	74	72.50%
Age	17-24	47	46.10%
	25-34	39	38.20%
	35+	16	15.70%
Frequency of Use	Rarely	15	14.70%
	Sometimes	20	19.60%
	Often	34	33.30%
	Always	33	32.40%
Device Type	Desktop	16	15.70%
	Laptop	41	40.20%
	Tablet	26	25.50%
	Smart Phone	19	18.60%

Regarding the frequency of interaction with ChatGPT, our data exhibits an intriguing spread. A significant chunk of participants (33.3%) reported using ChatGPT frequently, while 32.4% always resorted to the tool. On the other hand, 19.6% of respondents sometimes utilize AI, and a smaller portion (14.7%) rarely does. This information proposes a generally positive reliance on ChatGPT among translation students. Furthermore, it may indicate a growing trust in AI translation models’ effectiveness, accuracy, and viability in academic and professional translation tasks.

Finally, participants’ device preference data shows that the laptop was the most favoured choice (40.2%), followed by tablets (25.5%), smartphones (18.6%), and lastly, desktops (15.7%). This preference for portable devices (laptops, tablets, and smartphones) reinforces the prevailing trend of flexibility and mobility in digital tool usage. This is especially important in translation, where adaptability and access to digital resources-including AI translation models like ChatGPT-play a critical role. Unlike traditional translation-assisted software, which is often confined to either PC or laptop platforms, the newer tools, such as ChatGPT, introduce cross-device compatibility, allowing translators to work seamlessly on smartphones, tablets, and other portable devices. In turn, this could lead to increased productivity and enable translators to adapt to different environments.

The participant group had diverse English Language graduates with Arabic as their mother tongue. That made the insights regarding the perception and use of ChatGPT beneficial. Further, their academic backgrounds and experiences differed, and this helped enrich the study in understanding how AI translation models are received among different demographics and contexts of usage.

I.V.2. Quantitative Analysis

Interestingly, the participants’ responses to the 20 items in the 3-Point Likert Scale questionnaire, as described in Table 4, reveal some interesting insights concerning the participants’ experiences in handling the translation tasks set using ChatGPT. The “% agree,” “% neutral,” and “% disagree” columns denote the percentage of participants that responded positively, negatively, or neutrally to each item. Additionally, further quantitative insight is provided by the robust statistical measures in terms of “M (S.E.).” These supplement the qualitative understanding of the dataset.

Each of the 20 items in the questionnaire was meticulously curated to encapsulate the participants’ experiences and feedback regarding their interaction with the ChatGPT for translation purposes. The intricacies of these responses have been depicted comprehensively in Table 4.

Table 4 breaks down the percentage agreement along with the mean and standard error (M(S.E.)) for five key constructs, namely, Efficiency, Accuracy, Ease of Use, Trustworthiness, and Overall Preference.

Table 4. ChatGPT Performance and User Perceptions

No.	Construct	Item	M (S.E.)	% Agree	% Neutral	% Disagree
1.1		I can translate texts quickly using ChatGPT.	3.78 (1.1)	64.30%	26.20%	9.50%
1.2		ChatGPT helps me translate large volumes of text more easily.	3.61 (1.0)	58.70%	30.10%	11.20%
1.3	Efficiency	ChatGPT provides timely suggestions for translation.	3.51 (1.2)	52.80%	33.40%	13.80%
1.4		I can get my translation work done faster because of ChatGPT.	3.98 (0.8)	73.50%	22.70%	3.80%
2.1		ChatGPT usually provides accurate translations.	3.91 (0.9)	70.40%	26.90%	2.70%
2.2		The translations provided by ChatGPT do not require significant modifications.	3.49 (1.3)	48.70%	37.60%	13.70%
2.3	Accuracy	ChatGPT maintains the original meaning of the source text in the translated version.	3.62 (1.1)	54.60%	31.70%	13.70%
2.4		I can rely on the translations provided by ChatGPT for academic purposes.	3.74 (1.2)	62.90%	29.70%	7.40%
3.1		I find it easy to operate ChatGPT.	3.96 (1.0)	72.80%	24.50%	2.70%
3.2		The user interface of ChatGPT is user-friendly.	3.72 (1.1)	64.30%	28.20%	7.50%
3.3	Ease of Use	I find it easy to navigate through the features of ChatGPT.	3.84 (1.0)	67.60%	26.40%	6.00%
3.4		Learning to use ChatGPT is easy for me.	3.68 (1.2)	58.90%	32.10%	9.00%
4.1	Trustworthiness	I trust the translations provided by ChatGPT.	4.11 (1.0)	76.20%	17.60%	6.20%
4.2		ChatGPT is a reliable tool for translation tasks.	4.05 (0.7)	73.70%	21.20%	5.10%
4.3		I trust that ChatGPT will maintain the confidentiality of my translated work.	4.32 (1.1)	82.30%	10.70%	7.00%
4.4		I believe that the translations generated by ChatGPT are dependable.	3.84 (1.0)	65.40%	27.90%	6.70%
5.1	Overall Preference	I prefer using ChatGPT over other translation tools.	3.92 (1.2)	67.90%	22.70%	9.40%
5.2		I am satisfied with the performance of ChatGPT.	4.21 (0.8)	81.20%	16.80%	2.00%
5.3		I would recommend ChatGPT to other students.	4.08 (1.1)	75.40%	20.20%	4.40%
5.4		I will continue using ChatGPT for my future translation needs.	3.84 (1.0)	65.40%	27.90%	6.70%

Efficiency, the first construct, garnered considerable agreement among the participants. An overwhelming majority affirmed that ChatGPT enabled them to translate texts swiftly (64.3% agreement, M=3.78), thereby streamlining their workflow. The capability of ChatGPT to handle large volumes of text was also well-received (58.7% agreement, M=3.61), implying its suitability for extensive translation tasks. Participants further commended the timeliness of translation suggestions provided by ChatGPT (52.8% agreement, M=3.51), signifying its responsiveness to user inputs. Finally, ChatGPT’s role in accelerating translation tasks was acknowledged by a majority (73.5% agreement, M=3.98), reinforcing the tool’s contribution to productivity.

The second construct, accuracy, examined the quality of translations generated by ChatGPT. A substantial majority endorsed the accuracy of ChatGPT’s translations (70.4% agreement, M=3.91), indicating trust in the AI model’s linguistic competence. However, participants agreed to a lesser extent that translations provided by ChatGPT did not necessitate significant modifications (48.7% agreement, M=3.49), highlighting some room for improvement. ChatGPT’s ability to maintain the original meaning in its translations was appreciated (54.6% agreement, M=3.62), suggesting the model’s sensitivity to semantic preservation. Furthermore, students recognized the model as a reliable tool for academic translations (62.9% agreement, M=3.74), pointing to its adaptability to formal settings.

Ease of Use, the third construct, garnered favourable responses, with users finding ChatGPT simple to operate (72.8% agreement, M=3.96), underscoring its intuitiveness. The user-friendly interface was also appreciated (64.3% agreement, M=3.72), reflecting a positive user experience. In addition, the participants affirmed the ease of navigating through the features of ChatGPT (67.6% agreement, M=3.84), reflecting the tool’s ergonomic design. Moreover, the learning curve associated with using ChatGPT was gentle (58.9% agreement, M=3.68), indicating its accessibility for novice users.

Trustworthiness was highly regarded by participants who were specialists in English-Arabic translation and those with expertise in English-related fields like linguistics, literature, and language studies. The participants targeted students and professionals in translation and language disciplines to give an informed view of the subject matter. Most participants trusted the translations from ChatGPT; 76.2% agreed, M=4.11 hence trusted it. ChatGPT was recognized as a reliable tool for translation tasks (73.7% agreement, M=4.05), asserting its consistency in performance. The high level of trust in the tool’s confidentiality of translated work (82.3% agreement, M=4.32) illustrates the trustworthiness of ChatGPT, according to students, in handling sensitive information. Also, the dependability of the translations was affirmed by a majority (65.4% agreement, M=3.84), asserting the tool’s credibility.

The final construct, overall preference, revealed strong favourability towards ChatGPT. Participants preferred ChatGPT over other translation tools (67.9% agreement, M=3.92), indicating its competitive edge. Furthermore, an overwhelming majority expressed satisfaction with ChatGPT’s performance (81.2% agreement, M=4.21), reflecting a high degree of user contentment. Moreover, willingness to recommend ChatGPT to peers was high (75.4% agreement, M=4.08), indicating the tool’s influence and acceptance within the user community. Continued usage of ChatGPT for future translation needs was also anticipated (65.4% agreement, M=3.84), showing sustained interest and loyalty towards the tool.

These analytical insights into the five constructs provide a comprehensive understanding of ChatGPT’s perceived efficiency, accuracy, ease of use, trustworthiness, and overall participant preference. The in-depth examination of user experiences confirms ChatGPT’s potential as a highly regarded tool for translation purposes in academic contexts. The findings agree that ChatGPT is highly useful in Translation.

V. CONCLUSION

This study elicited translation students’ perceptions regarding using the tool in Translation. It has uncovered a significant breadth of insights. The findings, gathered through a quantitative method research design, elucidate the intricate user experiences with this sophisticated AI model. Furthermore, the study has examined several key constructs that determine the efficacy of such a tool, namely Efficiency, Accuracy, Ease of Use, Trustworthiness, and Overall Preference.

In the efficiency domain, ChatGPT performed very well, showing its capability in speeding up the translation process. Furthermore, the speed at which Translation of large volumes of text can be performed was highly appreciated by the participants, therefore increasing its utility in enhancing productivity. Accuracy is yet another important construct, and ChatGPT was noted for its capability to provide accurate translations despite the subtlety that there was room for improvement.

The ease of use of ChatGPT was another highlight of the study. Participants appreciated the model’s user-friendliness and intuitive design, indicating a favorable user experience. The learning curve associated with mastering the tool was also perceived to be gentle, signifying its potential as a beginner-friendly digital instrument to aid in translation. Trustworthiness emerged as another forte of ChatGPT, with users trusting the tool’s commitment to confidentiality and reliability. This finding was further bolstered by the participants’ preference for ChatGPT over other available translation platforms, which affirms its competitiveness in the landscape of AI-assisted Translation. Nonetheless, the study has its limitations. The predominantly young female demographic composition of the participant group poses a potential challenge to the broader generalizability of the findings. Future research could, therefore, benefit from including a more varied demographic sample to ensure a comprehensive representation of user perceptions of ChatGPT. The research context might only partially represent diverse professional translation scenarios. Hence, subsequent investigations could consider expanding the study environment to include professional translators and various translation contexts. This could pave the way for a holistic understanding of ChatGPT’s applicability across different professional settings.

The current study sets the stage for subsequent investigations into other language pairs, furthering the understanding of AI translation models’ potential in a more global context. Furthermore, given the ever-evolving landscape of AI and its consistent advancements, a longitudinal research approach could offer invaluable insights into changing user perceptions and adaptation practices. To summarize, this research provides a noteworthy contribution to the evolving literature on AI-assisted translation models, with a particular focus on ChatGPT. Furthermore, the results underscore the significance of integrating state-of-the-art digital tools into the translation course relating to MT and CAT tools, considering their increasing relevance and prevalence in the translation industry.

REFERENCES

Almahasees, Z. (2017). Machine Translation Quality of Khalil Gibran’s The Prophet (SSRN Scholarly Paper 3068518). https://doi.org/10.2139/ssrn.3068518

Almahasees, Z. (2018). Assessment of Google and Microsoft Bing translation of journalistic texts. International Journal of Languages, Literature, and Linguistics, 4(3), 231–235.

Almahasees, Z. (2021). Analysing English-Arabic Machine Translation: Google Translate, Microsoft Translator and Sakhr. Routledge. https://www.taylorfrancis.com/books/mono/10.4324/9781003191018/analysing-english-arabic-machine-translation-zakaryia-almahasees

Al-Salman, S., & Haider, A. (2024). Assessing the accuracy of MT and AI tools in translating humanities or social sciences Arabic research titles into English: Evidence from Google Translate, Gemini, and ChatGPT. International Journal of Data and Network Science, 8(4), 2483-2498.

Al-Taher, M. (2019). Google Translate’s rendition of verb-subject structures in Arabic news reports. International Journal of Arabic-English Studies, 19(1), 195-208.

Anazawa, R., Ishikawa, H., Park, M., & Kiuchi, T. (2013). Online machine translation use with nursing literature: Evaluation method and usability. Computers, Informatics, Nursing: CIN, 31(2), 59–65. https://doi.org/10.1097/NXN.0b013e3182701056

Archila, P.A., & de Mejía, A. (2020). Bilingual Teaching Practices in University Science Courses: How Do Biology and Microbiology Students Perceive Them? Journal of Language, Identity & Education, 19(3), 163–178. https://doi.org/10.1080/15348458.2019.1654868

Cai, D., Wang, Y., Li, H., Lam, W., & Liu, L. (2021). Neural Machine Translation with Monolingual Translation Memory (arXiv:2105.11269). arXiv. https://doi.org/10.48550/arXiv.2105.11269

Cronbach, L. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555

Deng, X., & Yu, Z. (2022). A Systematic Review of Machine-Translation-Assisted Language Learning for Sustainable Education. Sustainability, 14(13), Article 13. https://doi.org/10.3390/su14137598

Farghal, M., & Haider, A. (2024). Translating classical Arabic verse: human translation vs. AI large language models (Gemini and ChatGPT). Cogent Social Sciences, 10(1), 2410998.

Haleem, A., Javaid, M., & Singh, R.P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089. https://doi.org/10.1016/j.tbench.2023.100089

Hariri, W. (2024). Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing (arXiv:2304.02017). arXiv. https://doi.org/10.48550/arXiv.2304.02017

Hasyim, M., Saleh, F., Yusuf, R., & Abbas, A. (2021). Artificial Intelligence: Machine Translation Accuracy in Translating French-Indonesian Culinary Texts. Available at SSRN 3816594. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3816594

Jiao, W., Wang, W., Huang, J., Wang, X., Shi, S., & Tu, Z. (2023). Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine (arXiv:2301.08745). arXiv. https://doi.org/10.48550/arXiv.2301.08745

Jolley, J. , & Maimone, L. (2022). Thirty Years of Machine Translation in Language Teaching and Learning: A Review of the Literature. L2 Journal: An Open Access Refereed Journal for World Language Educators, 14(1). https://doi.org/10.5070/L214151760

Klimova, B., Pikhart, M., Benites, A.D., Lehr, C., & Sanchez-Stockhammer, C. (2023). Neural machine translation in foreign language teaching and learning: A systematic review. Education and Information Technologies, 28(1), 663–682. https://doi.org/10.1007/s10639-022-11194-2

Krings, H. (2001). Repairing texts: Empirical investigations of machine translation post-editing processes (Vol. 5). Kent State University Press. https://books.google.com/books?hl=en&lr=&id=vsdPsIXCiWAC&oi=fnd&pg=PP15&dq=Post+editing+holds+significant+promise+for+training+translators&ots=AcEDi5xJxh&sig=e2CjDycTGqqT7wPwYyYVh6Mrg2M

Lee, S-M. (2020). The impact of using machine translation on EFL students’ writing. Computer Assisted Language Learning, 33(3), 157–175. https://doi.org/10.1080/09588221.2018.1553186

Li, H., & Chen, H. (2019). Human vs. AI: An Assessment of the Translation Quality Between Translators and Machine Translation. International Journal of Translation, Interpretation, and Applied Linguistics (IJTIAL), 1(1), 1–12. https://doi.org/10.4018/IJTIAL.2019010104

Monib, W. K., Qazi, A., Apong, R. A., & Mahmud, M. M. (2024). Investigating learners’ perceptions of microlearning: Factors influencing learning outcomes. IEEE Access.

Nunnally, J. (1978). An Overview of Psychological Measurement. In B. B. Wolman (Ed.), Clinical Diagnosis of Mental Disorders (pp. 97–146). Springer US. https://doi.org/10.1007/978-1-4684-2490-4_4

O’Hagan, M. (2019). Introduction: Translation and technology: disruptive entanglement of human and machine. In The Routledge handbook of translation and technology (pp. 1–18). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315311258-1/introduction-minako-hagan

Peng, K., Ding, L., Zhong, Q., Shen, L., Liu, X., Zhang, M., Ouyang, Y., & Tao, D. (2023). Towards Making the Most of ChatGPT for Machine Translation (arXiv:2303.13780). arXiv. https://doi.org/10.48550/arXiv.2303.13780

Sujarwo, S. (2020). Students’ Perceptions of Using Machine Translation Tools In the EFL Classroom. Al-Lisan: Jurnal Bahasa (e-Journal), 5(2), Article 2. https://doi.org/10.30603/al.v6i2.1333

Received: 03 February 2024

Accepted: 18 December 2024