Re-examining Quality in AI-Mediated Language Learning: A PLS-SEM Analysis of Pedagogical Alignment, Engagement, and Arabic Language Proficiency

Hendro Risbiyantoro(1), Nunu Mahnun(2), Aprijon Efendi(3), Miftah Hur Rahman Zh(4), Muhibuddin Fadhli(5), Zahid Zufar At Thaariq(6), Aldo Redho Syam(7), Jimly Assiddiqi(8),

DOI: https://doi.org/10.46843/jpm.v5i1.709


(1) Universitas Darunnajah, Indonesia
(2) Universitas Islam Negeri Sultan Syarif Kasim, Indonesia
(3) Universitas Islam Negeri Sultan Syarif Kasim, Indonesia
(4) Universitas Darunnajah, Indonesia
(5) Universitas Negeri Malang, Indonesia
(6) Cukurova University, Turkey
(7) Universitas Negeri Jakarta, Indonesia
(8) Universitas Negeri Malang, Indonesia

Abstract

This study examines the effects of AI-based learning tools on Arabic language learning among Arabic students, with student engagement and educational content quality as mediators. This study used a quantitative causal-explanatory design and administered an online questionnaire to students, teachers, and education professionals (N=960). Using Partial Least Squares Structural Equation Modeling (PLS-SEM) to analyze the data, there was a large direct positive effect of AI tools on proficiency and an even stronger positive mediating effect of student engagement. Unexpectedly, perceptions of educational content quality were strongly negatively associated with proficiency, suggesting that a mismatch between content design and the learner’s sociocultural context may impede learning even when the pedagogical design is sound. The results highlight that the success of AI in language learning is not merely a function of the state of the technology, but, more importantly, a function of pedagogically sound content and its continued capacity to promote profound engagement. This research enhances the science of technology-enhanced learning by proposing a refined model for English language education in Egypt and by providing implementation-oriented suggestions for furthering practice, calling for a shift from technocentric implementation to learner-centric orchestration of AI in Arabic language education.

Keywords

Artificial Intelligence (AI), Arabic Language Proficiency, Student Engagement, Educational Content Quality, Technology-Enhanced Language Learning (TELL), and Pedagogical Alignment

Full Text:

PDF

References

Abed, M. G. (2025). The Role of Artificial Intelligence in Generating Grammatically Correct Arabic Texts. Journal of Educational Technology & Society, 28(1), 45–60.

Adeshola, I., & Adepoju, A. P. (2023). The Opportunities and Challenges of ChatGPT in Education. Interactive Learning Environments.

Ahmed, A. (2025). The Impact of Artificial Intelligence on Teaching Arabic Grammar. International Journal of Arabic Linguistics, 12(2), 89–104.

Ahmed, A., & Mahmoud, S. (2022). Beyond Delivery: The Pedagogical Design and Accuracy of AI-Generated Educational Content. Computers & Education, 185, 104521.

Ahmed, A., & Rahman, M. (2019). Personalized Arabic Language Learning Through AI-Driven Adaptive Systems. Journal of Language and Education, 5(3), 112–125.

Al-Ghamdi, A. (2020). The Role of AI in Mastering Nahwu and Sharaf: Challenges and Opportunities. Arabic Language Teaching Journal, 8(2), 134–150.

Al-Hamad, R. (2017). Technology-Enhanced Language Learning for Arabic: Effects on Speaking and Writing. Journal of Educational Computing Research, 55(4), 512–530.

Al-Mahmoud, F., Al-Refai, M., & Al-Said, K. (2022). AI-Enhanced Language Learning: A Meta-Analysis of Recent Studies in Arabic Education. Computers & Education, 187, 104541.

Al-Mahrooqi, R., & Denman, C. (2020). Assessing Arabic Language Proficiency: A Framework Based on International Standards. Language Testing in Asia, 10(1), 1–18.

Al-Mansoori, S. (2020). Socio-Linguistic Awareness in AI Tools for Arabic Language Learning. Computer Assisted Language Learning, 33(7), 789–810.

Albnian, M., Al-Shar’i, N., & Al-Shaboul, Y. (2025). Legal and Ethical Considerations in Deploying AI-Generated Digital Works in Arabic Language Education. International Review of Law, Computers & Technology, 39(1), 45–67.

Ali, M., Hussein, A., & Zawawi, I. (2020). AI Technology and Student Interaction in Arabic Language Learning. International Journal of Emerging Technologies in Learning, 15(10), 45–59.

Bati, T. (2024). Dataset Bias in AI-Generated Arabic Content: Implications for Educational Relevance. AI & Society, 39(2), 345–360.

Celik, I., Dindar, M., Muukkonen, H., & Järvelä, S. (2022). The Promises and Challenges of Artificial Intelligence for Teachers: A Systematic Review of Research. TechTrends, 66(4), 616–630.

Cheng, L., Wang, X., & Chen, Z. (2023). An Ecosystem Perspective on AI in Education: A Scoping Review. Computers & Education, 196, 104726.

Chin, W. W. (2008). The Partial Least Squares Approach to Structural Equation Modeling. In G. A. Marcoulides (Ed.), Modern Methods for Business Research (pp. 295–336). Lawrence Erlbaum Associates.

Chin, W. W. (2010). How to Write Up and Report PLS Analyses. In V. Esposito Vinzi, W. W. Chin, J. Henseler, & H. Wang (Eds.), Handbook of Partial Least Squares (pp. 655–690). Springer.

Chun, D., Kern, R., & Smith, B. (2016). Technology in Language Use, Language Teaching, and Language Learning. The Modern Language Journal, 100(S1), 64–80.

Creswell, J. W. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (5th ed.). Sage Publications.

Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319–340.

Dewaele, J.-M., & MacIntyre, P. D. (2014). The Two Faces of Janus? Anxiety and Enjoyment in the Foreign Language Classroom. Studies in Second Language Learning and Teaching, 4(2), 237–274.

Diamantopoulos, A., Sarstedt, M., Fuchs, C., Wilczynski, P., & Kaiser, S. (2012). Guidelines for Choosing Between Multi-Item and Single-Item Scales for Construct Measurement: A Predictive Validity Perspective. Journal of the Academy of Marketing Science, 40(3), 434–449.

Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models With Unobservable Variables and Measurement Error. Journal of Marketing Research, 18(1), 39–50.

Franke, G., & Sarstedt, M. (2019). Heuristics Versus Statistics in Discriminant Validity Testing: A Comparison of Four Procedures. Internet Research, 29(3), 430–447.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research, 74(1), 59–109.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education. The Internet and Higher Education, 2(2–3), 87–105.

Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM) (2nd ed.). Sage Publications.

Hair, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S. (2021). Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R: A Workbook. Springer.

Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to Use and How to Report the Results of PLS-SEM. European Business Review, 31(1), 2–24.

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A New Criterion for Assessing Discriminant Validity in Variance-Based Structural Equation Modeling. Journal of the Academy of Marketing Science, 43(1), 115–135.

Hwang, G.-J., Xie, H., Wah, B. W., & Gaševi?, D. (2020). Vision, Challenges, Roles and Research Issues of Artificial Intelligence in Education. Computers & Education: Artificial Intelligence, 1, 100001.

Ifenthaler, D., & Yau, J. Y.-K. (2020). Utilising Learning Analytics to Support Study Success in Higher Education: A Systematic Review. Educational Technology Research and Development, 68, 1961–1990.

Khalil, M., & Ebner, M. (2014). How Satisfied Are You With Your MOOC? – A Research Study on Interaction in Huge Online Courses. Proceedings of EdMedia+ Innovate Learning, 830–839.

Khan, A., & Ali, S. (2022). Engagement and Content Quality in AI-Mediated Learning: A Disjointed Examination. Journal of Educational Technology Systems, 50(4), 456–475.

Khan, M., Saleh, A., & Al-Mansoori, S. (2020). AI-Driven Interactivity and Student Engagement in Foreign Language Learning. International Journal of Artificial Intelligence in Education, 30, 567–589.

Kock, N. (2015). Common Method Bias in PLS-SEM: A Full Collinearity Assessment Approach. International Journal of E-Collaboration, 11(4), 1–10.

Kock, N., & Hadaya, P. (2018). Minimum Sample Size Estimation in PLS-SEM: The Inverse Square Root and Gamma-Exponential Methods. Information Systems Journal, 28(1), 227–261.

Kurniasih, D. D., Zh, M. H. R., Ayunisa, D. A., Maryono, M., Mustofa, K., & Siswanto, S. (2025). Analysis of Awareness and Confidence in Learning Outcomes With Students’ Academic Motivation: SEM Approach. Jurnal Inovasi Teknologi Pendidikan, 12(1), 58–67.

Kuswandi, D., Fadhli, M., Zh, M. H. R., Haditia, M., Sinaga, M. N. A., Thaariq, Z. Z. A., & Ardiansyah, A. (2025). Implementation of Personalized Approach in Video Editing Learning to Improve Digital Competency of 21st-Century Learners. JCP Jurnal Cahaya Pendidikan, 11(1).

Lowry, P. B., & Gaskin, J. (2014). Partial Least Squares (PLS) Structural Equation Modeling (SEM) for Building and Testing Behavioral Causal Theory: When to Choose It and How to Use It. IEEE Transactions on Professional Communication, 57(2), 123–146.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). McGraw-Hill.

Pallant, J. (2020). SPSS Survival Manual: A Step by Step Guide to Data Analysis Using IBM SPSS (7th ed.). Routledge.

Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common Method Biases in Behavioral Research: A Critical Review of the Literature and Recommended Remedies. Journal of Applied Psychology, 88(5), 879–903.

Popenici, S. A. D., & Kerr, S. (2017). Exploring the Impact of Artificial Intelligence on Teaching and Learning in Higher Education. Research and Practice in Technology Enhanced Learning, 12(1), 22.

Radomir, L., & Moisescu, O. I. (2019). Discriminant Validity of the Customer-Based Corporate Reputation Scale: Some Causes for Concern. Journal of Product & Brand Management, 29(4).

Sarstedt, M., Hair, J. F., Ringle, C. M., Thiele, K. O., & Gudergan, S. P. (2019). Progress in Partial Least Squares Structural Equation Modeling Use in Marketing Research in the Last Decade. Psychology & Marketing, 36(5), 395–431.

Smith, B., & Jones, A. (2021). User Resistance and Teacher Readiness in AI Integration for Language Learning. Educational Technology Research and Development, 69, 1347–1365.

Smith, J., & Johnson, L. (2015). Digital Platforms for Language Learning: Impacts on Motivation and Basic Skills. Language Learning & Technology, 19(3), 124–141.

Trizano-Hermosilla, I., & Alvarado, J. M. (2016). Best Alternatives to Cronbach’s Alpha Reliability in Realistic Conditions: Congeneric and Asymmetrical Measurements. Frontiers in Psychology, 7.

Usof, R., Mansor, N., & Salleh, N. (2025). Duolingo as a Tool for Teaching Arabic to Non-Native Speakers: Engagement and Proficiency Outcomes. International Journal of Mobile Learning and Organisation, 19(1), 55–78.

Xu, W., Ouyang, F., & Zhang, X. (2023). A Systematic Review of AI Role in Motivation and Learning. Educational Technology & Society, 26(1), 31–45.

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic Review of Research on Artificial Intelligence Applications in Higher Education – Where Are the Educators? International Journal of Educational Technology in Higher Education, 16(1), 39.

Zh, M. H. R. (2021). Implementation of the One Day Three Sentences Technique to Improve the Arabic Ability of Students at Modern Darussalam Gontor. Proceeding IConIGC: International Conference on Islamic and Global Civilization, 10–17.

Zh, M. H. R., Thaariq, Z. Z. A., & Ardiansyah, A. (2024). Mobile Learning: Future Learning Technologies for Islamic Formal Education (A Literature Study). Proceedings of International Conference on Education, 2(1), 370–377.

Zhang, D. (2021). Technological Access Disparities in AI-Enabled Education: A Global Perspective. Computers & Education, 173, 104292.

Zhang, D., & Nunamaker, J. F. (2003). Powering E-Learning in the New Millennium: An Overview of E-Learning and Enabling Technology. Information Systems Frontiers, 5(2), 207–218.

Refbacks

  • There are currently no refbacks.