The Application of ChatGPT in Language Test Design – The What and How

Authors

DOI:

https://doi.org/10.54855/paic.2348

Keywords:

Artificial intelligence (AI), ChatGPT, language test, test design

Abstract

ChatGPT, an artificial intelligence-based chatbot, has recently gone viral among users thanks to its ability to create human-like interactive experiences. It is now a trend to make use of this artificial intelligence (AI) tool in different fields of language education, including language testing and assessment. With a concern related to the potential and challenges of applying ChatGPT in designing language tests, this study has investigated the current use of the chatbot among language teachers at different schools and institutions. Participants are expected to share their experiences utilizing the chatbot to assess their learners. Data from a survey and in-depth interviews will reveal initial findings to answer questions related to the popularity of this chatbot among language teachers and its application in language test design. Based on the result discussion, the presentation concludes with some implications for language teachers to make the most of their future use of ChatGPT in the field of language testing.

Author Biography

Nguyen Thi Phuong Thao, University of Languages and International Studies, Vietnam National Universities, Ha noi, Vietnam

Nguyen Thi Phuong Thao (Thao Nguyen) is a lecturer and researcher at VNU University of Languages and International Studies. She earned a Master’s degree in English Language Teaching from the University of Southampton, UK. She has participated in research projects on test development and examiner training. Her research interests include testing and assessment, teacher education, and professional development.

References

Cotton, D.R., Cotton, P.A., & Shipway, J.R. (2023). Chatting and cheating. Ensuring academic integrity in the era of ChatGPT. https://doi.org/10.1080/14703297.2023.2190148

Choi, J.H., Hickman, K.E., & Monahan, A.B. (2023). ChatGPT goes to law school. https://dx.doi.org/10.2139/ssrn.4335905

Dao, X.Q., Le, N.B., Phan, X.D, & Ngo, B.B. (2023). Can ChatGPT pass the Vietnamese National High School Graduation Examination? http://dx.doi.org/10.48550/arXiv.2306.09170

Hong, W.S.H, (2023). The impact of ChatGPT on foreign language teaching and learning: opportunities in education and research. Journal of Education and Technology and Innovation, 5(1), 37-45.

E.Kasneci et al. (2023) ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences. 103. https://doi.org/10.1016/j.lindif.2023.102274.

Le, A.V., Bui, T.D., Le, Q.Q., & Vu, V.L. (2023). Khả năng thực hiện bài kiểm tra định kì môn Toán và môn Ngữ văn cấp Trung học của công cụ ChatGPT: Kết quả nghiên cứu và một số khuyến nghị ban đầu. Tạp chí Khoa học và Giáo dục Việt Nam, 19(2), 1-10.

Moore, N.S., & MacArthur, C.A. (2016). Student use of automated essay evaluation technology during revision. Journal of Writing Research, 8(1), 149-175.

Nguyen, Q. H. (2023). AI and Plagiarism: Opinion from Teachers, Administrators and Policymakers. Proceedings of the AsiaCALL International Conference, 4, 75–85. https://doi.org/10.54855/paic.2346.

Nisar, S., & Aslam, M.S. (2023). Is ChatGPT a good tool for T&CM students in studying pharmacology? Available at http://dx.doi.org/10.2139/ssrn.4324310

Raford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Retrieved from

http://www.cs.ubc.ca/amuham01/LING530/papers/radford2018improving.pdf

Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of the traditional assessment in higher education. Journal of Applied Learning & Teaching, 6(1), 242-263. Available at https://journals.sfu.ca/jalt/index.php/jalt/index

Tate, T., Doroudi, S., Ritchie, D., & Xu, Y. (2023). Educational research and AI-generated writing: Confronting the coming tsunami. Available at https://doi.org/10.35542/osf.io/4mec3.

TESOL International association. (2023). AI for Language Learning: ChatGPT and the Future of ELT. Available at http://blog.tesol.org/ai-for-language-learning-chatgpt-and-the-future-of-elt/

Thai, T.C.T. (2023). Thái độ và kì vọng của sinh viên sư phạm tiếng Anh đối với ChatGPT: Nghiên cứu tại trường Đại học Sư phạm Hà Nội. Tạp chí Giáo dục, 23(10), 51-61.

Tran, T.N., & Tran, H.P. (2023). Exploring the role of ChatGPT in developing critical digital literacies in language learning: a qualitative study. Proceedings of the AsiaCALL International Conference, 4, 45-62. https://doi.org/10.54855/paic.2344.

Yeadon, W., Inyang, O.O., Mizouri, A., Peach, A., & Testrow, C. (2022). The death of the short-form physics essay in the coming AI revolution. Physics Education, 58(3). https://iopscience.iop.org/article/10.1088/1361-6552/acc5cf/pdf

Zhai, X. (2022). ChatGPT user experience: Implications for education. Available at http://dx.doi.org/10.2139/ssrn.4312418.

Downloads

Published

2023-12-31

How to Cite

Nguyen, T. P. T. (2023). The Application of ChatGPT in Language Test Design – The What and How . Proceedings of the AsiaCALL International Conference, 4, 104–115. https://doi.org/10.54855/paic.2348

Issue

Vol. 4 (2023): Proceedings of the 20th AsiaCALL International Conference (AsiaCALL2023)

Section

Research Article

License

Copyright (c) 2023 Nguyen Thi Phuong Thao

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The copyright of all articles published in the Proceedings of the AsiaCALL International Conference (paic) remains with the Authors, i.e. Authors retain full ownership of their article. Permitted third-party reuse of the open access articles is defined by the applicable Creative Commons (CC) end-user license which is accepted by the Authors upon submission of their paper. All articles in the aicp are published under the CC BY-NC 4.0 license, meaning that end users can freely share an article (i.e. copy and redistribute the material in any medium or format) and adapt it (i.e. remix, transform and build upon the material) on the condition that proper attribution is given (i.e. appropriate credit, a link to the applicable license and an indication if any changes were made; all in such a way that does not suggest that the licensor endorses the user or the use) and the material is only used for non-commercial purposes. 

Most read articles by the same author(s)