Evaluating HUFLIT Lecturers’ Perspectives on ChatGPT’s Capabilities in Designing English Testing and Assessment
DOI:
https://doi.org/10.54855/paic.24612Keywords:
ChatGPT’s capabilities, testing and assessment, lecturers’ perspectives, HUFLITAbstract
Despite its widespread adoption in education, the potential of ChatGPT in English language teaching and learning, especially in testing and assessment design, still needs to be studied. This paper, therefore, seeks to investigate lecturers’ perspectives on the capabilities of ChatGPT in designing English testing and assessment with the participation of thirty lecturers of the Faculty of Foreign Languages at HUFLIT. Data from the questionnaire and semi-structured interview suggest that ChatGPT is a promising tool for designing English testing and assessment owing to its versatility in drafting quicker, more diverse, and more detailed types of questions, resulting in enhanced time efficiency and reduced efforts in test design. The greatest areas of concern, however, lie in ChatGPT's inability to generate questions that match the specific learning outcomes and students' linguistic abilities. The study contributes to providing lecturers with in-depth insights into the potentialities and challenges of ChatGPT in testing and assessment design, assisting them in employing this AI tool more critically and effectively.
References
Abas, M. A., Arumugam, S. E., Yunus, M. M., & Rafiq, K. R. (2023). ChatGPT and Personalized Learning: Opportunities and Challenges in Higher Education. International Journal of Academic Research in Business and Social Sciences, 13(12), 3536-3545. DOI: https://doi.org/10.6007/IJARBSS/v13-i12/20240
Ahmed, F., Ali, S., & Shah, R. A. (2019). Exploring Variation in Summative Assessment: Language Teachers’ Knowledge of Students' Formative Assessment and Its Effect on their Summative Assessment. Bulletin of Education and Research, 41(2), 109-119.
Almasre, M. (2024). Development and Evaluation of a Custom GPT for the Assessment of Students’ Designs in a Typography Course. Education Sciences, 14, 1-19. https://doi.org/10.3390/educsci14020148 DOI: https://doi.org/10.3390/educsci14020148
Amin, M. Y. (2023). AI and Chat GPT in Language Teaching: Enhancing EFL Classroom Support and Transforming Assessment Techniques. International Journal of Higher Education Pedagogies, 4(4), 1-15. Retrieved from https://doi.org/10.33422/ijhep.v4i4.554 DOI: https://doi.org/10.33422/ijhep.v4i4.554
Baskara, F. R., & Mukarto, F. (2023). Exploring the Implications of ChatGPT for Language Learning in Higher Education. Indonesian Journal of English Language Teaching and Applied Linguistics, 7(2), 343-358.
Beikmahdavi, N. (2016). Washback in Language Testing: Review of Related Literature First. International Journal of Modern Language Teaching and Learning, 1(4), 130-136.
Black, P., & Jones, J. (2006). Formative assessment and the learning and teaching of MFL: sharing the language learning road map with learners. Language Learning Journal(34), 4-9. DOI: https://doi.org/10.1080/09571730685200171
Brown, G. T. (2004). Teachers' conceptions of assessment: Implications for policy and professional development. Assessment in Education: Principles, Policy & Practice, 301-318. https://doi.org/10.1080/0969594042000304609 DOI: https://doi.org/10.1080/0969594042000304609
Chandio, M. T., & Jafferi, S. (2015). Teaching English as a Language not Subject by Employing Formative Assessment. Journal of Education and Educational Development, 2(2), 151-171. DOI: https://doi.org/10.22555/joeed.v2i2.444
Fadilah, R., Ayudhia, H. Y., Chairani, V. S., & Afni, F. (2023). Assessment of English Language Teaching for 21st Century: Teachers’ Perspectives on Traditional and Alternative Assessment. JADEs Journal of Academia in English Education, 4(1), 108-130. Retrieved from https://journal.iainlangsa.ac.id/index.php/jades DOI: https://doi.org/10.32505/jades.v4i1.6023
Heaven, W. D. (2023, April 7). ChatGPT is going to change education, not destroy it. MITTechnology Review. Retrieved from https://www.technologyreview.com/2023/04/06/1071059/-chatgpt-
Jain, N. (2021). Survey Versus Interviews: Comparing Data Collection Tools for Exploratory Research. The Qualitative Report, 26(2), 541-554. Retrieved from https://doi.org/10.46743/2160-3715/2021.4492 DOI: https://doi.org/10.46743/2160-3715/2021.4492
Kolade, O., Owoseni, A., & Egbetokun, A. (2024). Is AI changing learning and assessment as we know it? Evidence from a ChatGPT experiment and a conceptual framework. Heliyon, 10, 1-21. DOI: https://doi.org/10.1016/j.heliyon.2024.e25953
Koraishi, O. (2023). Teaching English in the Age of AI: Embracing ChatGPT to Optimize EFL Materials and Assessment. Language Education & Technology(LET Journal), 3(1), 55-72.
Kostka, I., & Toncelli, R. (2023). Exploring Applications of ChatGPT to English Language Teaching: Opportunities, Challenges, and Recommendations. The Electronic Journal for English as a Second Language, 27(3), 1-19. DOI: https://doi.org/10.55593/ej.27107int
Kuphanga, D. (2024). Questionnaires in Research: Their Role, Advantages, and Main Aspects. Research Gate, 1-8. https://doi.org/10.13140/RG.2.2.15334.64325
Mansory, M. (2020). The Significance of Non-traditional and Alternative Assessment in English Language Teaching: Evidence From Literature. International Journal of Linguistics, 12(5), 210-225. DOI: https://doi.org/10.5296/ijl.v12i5.17782
Moqbel, M. S., & Al-Kadi, A. M. (2023). Foreign Language Learning Assessment in the Age of ChatGPT: A Theoretical Account. Journal of English Studies in Arabia Felix, 2(1), 71–84. https://doi.org/10.56540/jesaf.v2i1.62 DOI: https://doi.org/10.56540/jesaf.v2i1.62
Nguyen, T. P. T. (2023). The Application of ChatGPT in Language Test Design – The What and How. Proceedings of the AsiaCALL International Conference, 4, 104-115. ISSN: 2833-6836; ISBN: 979-8-9870112-4-9. DOI: https://doi.org/10.54855/paic.2348 DOI: https://doi.org/10.54855/paic.2348
Pham, V. P. H., & Le, A. Q. (2024). ChatGPT in Language Learning: Perspectives from Vietnamese Students in Vietnam and the USA. International Journal of Language Instruction, 3(2), 59-72. DOI: https://doi.org/10.54855/ijli.24325 DOI: https://doi.org/10.54855/ijli.24325
Rasul, T., Nair, S., Kalendra, D., Robin, M., Oliveira, F. d., Ladeira, W. J., . . . Heathcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning & Teaching, 6(1), 1-16. DOI: https://doi.org/10.37074/jalt.2023.6.1.29
Roumeliotis, K. I., & Tselikas, N. D. (2023). ChatGPT and Open-AI Models: A Preliminary Review. Future Internet, 15, 1-24. DOI: https://doi.org/10.3390/fi15060192
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), 342-363. DOI: https://doi.org/10.37074/jalt.2023.6.1.9
Trumbull, E., & Lash, A. (2013). Understanding Formative Assessment: Insights from Learning Theory and. San Francisco: WestEd.
Widiastuti, I. A., & Saukah, A. (2017). Formative Assessment in EFL Classroom Practices. Bahasa dan Seni: Jurnal Bahasa, Sastra, Seni, dan Pengajarannya, 45(1), 50-63. https://doi.org/10.17977/um015v45i12017p050 DOI: https://doi.org/10.17977/um015v45i12017p050
Zaiarna, I., Zhyhadlo, O., & Dunaievska, O. (2024). ChatGPT In Foreign Language Teaching And Assessment: Exploring EFL Instructors’ Experience. Information Technologies and Learning Tools, 102(4), 176-191. https://doi.org/10.33407/itlt.v102i4.5716 DOI: https://doi.org/10.33407/itlt.v102i4.5716
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Le Thi Thu Huong
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright
The copyright of all articles published in the Proceedings of the AsiaCALL International Conference (paic) remains with the Authors, i.e. Authors retain full ownership of their article. Permitted third-party reuse of the open access articles is defined by the applicable Creative Commons (CC) end-user license which is accepted by the Authors upon submission of their paper. All articles in the aicp are published under the CC BY-NC 4.0 license, meaning that end users can freely share an article (i.e. copy and redistribute the material in any medium or format) and adapt it (i.e. remix, transform and build upon the material) on the condition that proper attribution is given (i.e. appropriate credit, a link to the applicable license and an indication if any changes were made; all in such a way that does not suggest that the licensor endorses the user or the use) and the material is only used for non-commercial purposes.
