Evaluation of Problem Based Gamification Learning (PBGL) Model on Critical Thinking Ability with Artificial Intelligence Approach Integrated with ChatGPT API: An Experimental Study

Remerta Noni Naatonis; Rusijono Rusijono; Miftakhul Jannah; Edwin Ariesto Umbu Malahina

doi:10.48161/qaj.v4n3a919

Authors

Remerta Noni Naatonis Educational Technology Study Program, State University of Surabaya, Surabaya City, 60213, Indonesia
Rusijono Educational Technology Study Program, State University of Surabaya, Surabaya City, 60213, Indonesia
Miftakhul Jannah Educational Technology Study Program, State University of Surabaya, Surabaya City, 60213, Indonesia
Edwin Ariesto Umbu Malahina Informatics Engineering Study Program, STIKOM Uyelindo Kupang, Kupang City, 85111, Indonesia

DOI:

https://doi.org/10.48161/qaj.v4n3a919

Keywords:

Gamification, Critical Thinking, ChatGPT, Artificial Intelligence, PBGL

Abstract

This research evaluates the effectiveness of the PBGL model integrated with Artificial Intelligence (A.I) using the ChatGPT API to improve students' critical thinking skills. The main problems faced are the limited time of the lecturer in delivering the material, the lack of reference material, and the low motivation of students. This research used an experimental method involving 520 students from Timor Leste (Cova-Lima and Dili districts) and Indonesia (East Java and East Nusa Tenggara provinces) studying Python, Java, and Web programming. Participants were divided into control (n=260) and experimental (n=260) groups. Key challenges included integration of the ChatGPT API, implementation of A.I-based automated feedback, and maintaining consistency and fairness in testing. The evaluation results showed significant improvement in critical thinking skills in the experimental group across all topics. For the Python topic, the experimental group had an average improvement of 20.75 points, compared to 8.20 points in the control group, with an average difference of 12.55 points. On the Java topic, the experimental group increased by an average of 25.58 points, while the control group only 7.86 points, with an average difference of 17.72 points. In the Web Programming Languages topic, the experimental group showed an average increase of 23.74 points, compared to 8.58 points in the control group, with an average difference of 15.16 points. These findings confirm that the contribution of A.I and ChatGPT integration in the PBGL model can improve critical thinking, provide automatic feedback, and increase student motivation. This research has the potential to be a reference in digital curriculum design with generative A.I approaches such as ChatGPT API, showing that A.I can enrich learning materials, personalized materials and provide flexibility for students and teachers in the digital era.

Downloads

Download data is not yet available.

References

Čubela, D., Rossner, A., & Neis, P. (2023). Using problem-based learning and gamification as a catalyst for student engagement in data-driven engineering education: A report. Educ Sci (Basel), 13(12), 1223–1235.

Daba, J. B. R., Rosmansyah, Y., & Dabarsyah, B. (2019). Problem based learning using gamification: A systematic literature review. In 2019 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) (pp. 125-130). IEEE.

Barua, A. M., & Bharali, S. S. (2023). Gamification and its challenges in e-learning: A case study of computer science learners at KKHSOU. Asian Association of Open Universities Journal, 18(3), 233-245.

Taesotikul, T., Chinpaisal, C., & Nawanopparatsakul, S. (2021). Kahoot! gamification improves learning outcomes in problem-based learning classroom. In 2021 3rd International Conference on Modern Educational Technology (pp. 125-129). ACM.

Kladchuen, R., & Srisomphan, J. (2021). The synthesis of a model of problem-based learning with the gamification concept to enhance the problem-solving skills for high vocational certificate. International Journal of Emerging Technologies in Learning (iJET), 16(14), 4-21.

Poonsawad, A., Srisomphan, J., & Sanrach, C. (2022). Synthesis of problem-based interactive digital storytelling learning model under gamification environment promotes students' problem-solving skills. International Journal of Emerging Technologies in Learning (iJET), 17(05), 103-119.

Imran, H. (2023). An empirical investigation of the different levels of gamification in an introductory programming course. Journal of Educational Computing Research, 61(4), 847-874.

Gejandran, P., & Abdullah, N. (2024). Gamification in e-learning: A systematic review of benefits, challenges, and future possibilities. Journal of Logistics, Informatics and Service Science, 11(2), 84-104.

Swacha, J. (2022). Topic evolution in the research on educational gamification. Educ Sci (Basel), 12(10), 640-653.

Vijay, M., & Jayan, J. P. (2023). Evolution of research on adaptive gamification: A bibliometric study. In 2023 International Conference on Circuit Power and Computing Technologies (ICCPCT) (pp. 1062-1070). IEEE.

Dichev, C., & Dicheva, D. (2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14(1), 9-24.

Oliveira, W., Hamari, J., Shi, L., Toda, A. M., Rodrigues, L., Palomino, P. T., & Isotani, S. (2023). Tailored gamification in education: A literature review and future agenda. Educ Inf Technol (Dordr), 28(1), 373-406.

Koravuna, S., & Surepally, U. K. (2020). Educational gamification and artificial intelligence for promoting digital literacy. In Proceedings of the 2nd International Conference on Intelligent and Innovative Computing Applications (pp. 1-6). ACM.

Bennani, S., Maalel, A., & Ben Ghezala, H. (2022). Adaptive gamification in e-learning: A literature review and future challenges. Computer Applications in Engineering Education, 30(2), 628-642.

Elbanna, S., & Armstrong, L. (2024). Exploring the integration of ChatGPT in education: Adapting for the future. Management & Sustainability: An Arab Review, 3(1), 16-29.

Rejeb, A., Rejeb, K., Appolloni, A., Treiblmaier, H., & Iranmanesh, M. (2024). Exploring the impact of ChatGPT on education: A web mining and machine learning approach. The International Journal of Management Education, 22(1), 1-14.

AlAli, R., & Wardat, Y. (2024). Enhancing classroom learning: ChatGPT's integration and educational challenges. International Journal of Religion, 5(6), 971-985.

Castonguay, A., Farthing, P., Davies, S., Vogelsang, L., Kleib, M., Risling, T., & Green, N. (2023). Revolutionizing nursing education through AI integration: A reflection on the disruptive impact of ChatGPT. Nurse Educ Today, 129(10), 105916–105929.

Angelelli, C. V., Ribeiro, G. M. de C., Severino, M. R., Johnstone, E., Borzenkova, G., & da Silva, D. C. O. (2023). Developing critical thinking skills through gamification. Think Skills Creat, 49(1), 101354–101366.

Rusandi, M. A., Ahman, Saripah, I., Khairun, D. Y., & Mutmainnah. (2023). No worries with ChatGPT: Building bridges between artificial intelligence and education with critical thinking soft skills. J Public Health (Bangkok), 45(3), e602–e603.

Supnoon, A., & Chonchaiya, R. (2024). A study on the development of eleventh grade students' critical thinking skills and self-efficacy using active learning pedagogy with gamification. International Journal of Education and Practice, 12(2), 447-466.

Heliawati, L., Lidiawati, L., & Pursitasari, I. D. (2022). Articulate Storyline 3 multimedia based on gamification to improve critical thinking skills and self-regulated learning. International Journal of Evaluation and Research in Education (IJERE), 11(3), 1435–1444.

Jodoi, K., Takenaka, N., Uchida, S., Nakagawa, S., & Inoue, N. (2021). Developing an active-learning app to improve critical thinking: Item selection and gamification effects. Heliyon, 7(11), 8256–8263.

Tzelepi, M., Makri, K., Petroulis, I., Moundridou, M., & Papanikolaou, K. (2020). Gamification in online discussions: How do game elements affect critical thinking? In 2020 IEEE 20th International Conference on Advanced Learning Technologies (ICALT) (pp. 92-94). IEEE.

Huang, L. Y., & Yeh, Y. C. (2017). Meaningful gamification for journalism students to enhance their critical thinking skills. International Journal of Game-Based Learning, 7(2), 47-62.

Mahdi, H. S., & Sahari, Y. M. (2024). Exploring the relationship between critical thinking, attitude, and anxiety in shaping the adoption of artificial intelligence in translation among Saudi translators. Journal of Pedagogical Research, 8(2), 81-94.

Nappi, C., & Cuocolo, A. (2020). The machine learning approach: Artificial intelligence is coming to support critical clinical thinking. Journal of Nuclear Cardiology, 27(1), 156-158.

Kuchyn, I. L., Lymar, L. V., Bielka, K. Y., Storozhuk, K. V., & Kolomiiets, T. V. (2024). New training, new attitudes: Non-clinical components in Ukrainian medical PhD training (regarding critical thinking, academic integrity, and artificial intelligence use). Wiadomości Lekarskie, 77(4), 665-669.

Jia, X. H., & Tu, J. C. (2024). Towards a new conceptual model of AI-enhanced learning for college students: The roles of artificial intelligence capabilities, general self-efficacy, learning motivation, and critical thinking awareness. Systems, 12(3), 74.

Kuhn, D. (2019). Critical thinking as discourse. Hum Dev, 62(3), 146-164.

Prokop-Dorner, A., Piłat-Kobla, A., Ślusarczyk, M., Świątkiewicz-Mośny, M., Ożegalska-Łukasik, N., Potysz-Rzyman, A., Zarychta, M., Juszczyk, A., Kondyjowska, D., Magiera, A., Maraj, M., Storman, D., Warzecha, S., Węglarz, P., Wojtaszek-Główka, M., Żabicka, W., & Bała, M. M. (2024). Teaching methods for critical thinking in health education of children up to high school: A scoping review. PLoS One, 19(7), e0307094–e0307094.

Shafieieh, M., Ozturen, A., Rezapouraghdam, H., & Karatepe, O. M. (2024). Does critical thinking mediate the relationship between sustainability knowledge and tourism students' ability to make sustainable decisions? Sustainability, 16(13), 5655–5671.

Butler, H. A. (2024). Predicting everyday critical thinking: A review of critical thinking assessments. J Intell, 12(2), 16-29.

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education - where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39-66.

Orji, C. T., & Ogbuanya, T. C. (2022). Mediating roles of ability beliefs and intrinsic motivation in PBL and engagement in practical skills relations among electrical/electronic education undergraduate. Innovations in Education and Teaching International, 59(3), 326-336.

Farikhatul Mutma'innah, & Hamimi, E. (2024). Development of PBL-based GLOWASEA (Global Warming on the Sea) educational media to train critical thinking skills on the topic of global warming. International Journal of Interactive Mobile Technologies (iJIM), 18(09), 117-140.

Fajari, L. E. W., Sarwanto, & Chumdari. (2020). Improving elementary school's critical thinking skills through three different PBL-assisted learning media viewed from learning styles. Journal of E-Learning and Knowledge Society, 16(1), 55-64.

Crespí, P., García-Ramos, J. M., & Queiruga-Dios, M. (2022). Project-based learning (PBL) and its impact on the development of interpersonal competences in higher education. Journal of New Approaches in Educational Research, 11(2), 259-276.

Azer, S. A. (2004). Twelve tips for becoming a student in a PBL course: Twelve tips for successful group discussion. Med Teach, 26(1), 12-15.

Loyens, S. M. M., Magda, J., & Rikers, R. M. J. P. (2008). Self-directed learning in problem-based learning and its relationships with self-regulated learning. Educ Psychol Rev, 20(4), 411-427.

Afzaal, M., Nouri, J., Zia, A., Papapetrou, P., Fors, U., Wu, Y., Li, X., & Weegar, R. (2021). Explainable AI for data-driven feedback and intelligent action recommendations to support students self-regulation. Front Artif Intell, 4(1), 1-20.

Ruiz-Rojas, L. I., Salvador-Ullauri, L., & Acosta-Vargas, P. (2024). Collaborative working and critical thinking: Adoption of generative artificial intelligence tools in higher education. Sustainability, 16(13), 5367–5390.

Wong, R. S. Y. (2024). ChatGPT in medical education: Promoting learning or killing critical thinking? Education in Medicine Journal, 16(2), 177-183.

Dahalan, F., Alias, N., & Shaharom, M. S. N. (2024). Gamification and game based learning for vocational education and training: A systematic literature review. Educ Inf Technol (Dordr), 29(2), 1279–1317.

Zeybek, N., & Saygı, E. (2024). Gamification in education: Why, where, when, and how—a systematic review. Games Cult, 19(2), 237-264.

Dehghanzadeh, H., Farrokhnia, M., Dehghanzadeh, H., Taghipour, K., & Noroozi, O. (2024). Using gamification to support learning in K-12 education: A systematic literature review. British Journal of Educational Technology, 55(1), 34-70.

Castellano, M. S., Contreras-McKay, I., Neyem, A., Farfán, E., Inzunza, O., Ottone, N. E., del Sol, M., Alario-Hoyos, C., Alvarado, M. S., & Tubbs, R. S. (2024). Empowering human anatomy education through gamification and artificial intelligence: An innovative approach to knowledge appropriation. Clinical Anatomy, 37(1), 12-24.

Bachiri, Y. A., Mouncif, H., & Bouikhalene, B. (2023). Artificial Intelligence empowers gamification: Optimizing student engagement and learning outcomes in e-learning and MOOCs. International Journal of Engineering Pedagogy (iJEP), 13(8), 4-19.

Freire-Palacios, V., Jaramillo-Galarza, K., Quito-Calle, J., & Orozco-Cantos, L. (2023). La inteligencia artificial en la gamificación para promover la salud mental de los estudiantes universitarios: una revisión de alcance. Salud, Ciencia y Tecnología, 3(1), 639-656.

Alahmari, M., Jdaitawi, M. T., Rasheed, A., Abduljawad, R., Hussein, E., Alzahrani, M., & Awad, N. (2023). Trends and gaps in empirical research on gamification in science education: A systematic review of the literature. Contemp Educ Technol, 15(3), ep431–ep447.

Smiderle, R., Rigo, S. J., Marques, L. B., Peçanha de Miranda Coelho, J. A., & Jaques, P. A. (2020). The impact of gamification on students' learning, engagement and behavior based on their personality traits. Smart Learning Environments, 7(1), 3-15.

Chan, K. S., & Zary, N. (2019). Applications and challenges of implementing artificial intelligence in medical education: An integrative review. JMIR Medical Education, 5(1), e13930–e13945.

Bachiri, Y. A., & Mouncif, H. (2023). Artificial Intelligence system in aid of pedagogical engineering for knowledge assessment on MOOC platforms: Open EdX and Moodle. International Journal of Emerging Technologies in Learning (iJET), 18(05), 144-160.

Naidu, K., & Sevnarayan, K. (2023). ChatGPT: An ever-increasing encroachment of artificial intelligence in online assessment in distance education. Online Journal of Communication and Media Technologies, 13(3), e202336–e202347.

Tsai, Y. C. (2023). Empowering learner-centered instruction: Integrating ChatGPT Python API and Tinker learning for enhanced creativity and problem-solving skills. In Innovative Technologies and Learning (1st ed., pp. 531-541). Cham: Springer.

Резаев, А. В., & Трегубова, Н. Д. (2023). От социологии алгоритмов к социальной аналитике искусственной социальности: анализ кейсов API и ChatGPT. The Monitoring of Public Opinion: Economic & Social Changes, 3(3), 3-22.

Babu, S. S., & Moorthy, A. D. (2024). Application of artificial intelligence in adaptation of gamification in education: A literature review. Computer Applications in Engineering Education, 32(1), e22683–e22698.

Yu, P., & Wang, S. (2024). An examination and analysis of the integration of artificial intelligence and gamification in the pedagogy of Chinese higher education. In Engaged Learning and Innovative Teaching in Higher Education (1st ed., pp. 29-46). Singapore: Springer.

Modi, T., & Gochhait, S. (2023). Impact of artificial intelligence on gamification: Current applications. In 2023 International Conference on Innovative Data Communication Technologies and Application (ICIDCA) (pp. 287-290). Uttarakhand: IEEE.

van den Berg, G., & du Plessis, E. (2023). ChatGPT and generative AI: Possibilities for its contribution to lesson planning, critical thinking, and openness in teacher education. Educational Sciences (Basel), 13(10), 998–1010.

Alarcón-López, C., Krütli, P., & Gillet, D. (2024). Assessing ChatGPT's influence on critical thinking in sustainability-oriented activities. In 2024 IEEE Global Engineering Education Conference (EDUCON) (pp. 1-10). Kos Island, Greece: IEEE.

Pishtari, G., Sarmiento-Márquez, E., Rodríguez-Triana, M. J., Wagner, M., & Ley, T. (2024). Mirror mirror on the wall, what is missing in my pedagogical goals? The impact of an AI-driven feedback system on the quality of teacher-created learning designs. In Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 145-156). New York, NY, USA: ACM.

Khaldi, A., Bouzidi, R., & Nader, F. (2023). Gamification of e-learning in higher education: A systematic literature review. Smart Learning Environments, 10(1), 10-21.

Kopp, F., & Mendell, J. T. (2018). Functional classification and experimental dissection of long noncoding RNAs. Cell, 172(3), 393-407.

Lv, B. Q., Weng, H. M., Fu, B. B., Wang, X. P., Miao, H., Ma, J., Richard, P., Huang, X. C., Zhao, L. X., Chen, G. F., Fang, Z., Dai, X., Qian, T., & Ding, H. (2015). Experimental discovery of Weyl semimetal TaAs. Physical Review X, 5(3), 031013–031026.

Sailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior, 69(12), 371-380.

Baghestani, A. R., Ahmadi, F., Tanha, A., & Meshkat, M. (2019). Bayesian critical values for Lawshe's content validity ratio. Measurement and Evaluation in Counseling and Development, 52(1), 69-73.

Alavi, M., Biros, E., & Cleary, M. (2024). Notes to factor analysis techniques for construct validity. Canadian Journal of Nursing Research, 56(2), 164-170.

Gamarra, M., Dominguez, A., Velazquez, J., & Páez, H. (2022). A gamification strategy in engineering education: A case study on motivation and engagement. Computer Applications in Engineering Education, 30(2), 472-482.

Sotos-Martínez, V. J., Ferriz-Valero, A., García-Martínez, S., & Tortosa-Martínez, J. (2024). The effects of gamification on the motivation and basic psychological needs of secondary school physical education students. Physical Education and Sport Pedagogy, 29(2), 160-176.

Gkintoni, E., Vantaraki, F., Skoulidi, C., Anastassopoulos, P., & Vantarakis, A. (2024). Promoting physical and mental health among children and adolescents via gamification: A conceptual systematic review. Behavioral Sciences, 14(2), 102-134.

Monroe, C. M., Zosel, K., Stansbury, M., Younginer, N., Davis, R. E., Dutton, G., Newton Jr., R. L., Cai, B., & West, D. S. (2024). A focus group study among insufficiently physically active African American adults regarding technology-delivered team-based gamification for physical activity promotion. mHealth, 10(1), 3-3.

Torres-Toukoumidis, A., León, D. V., De-Santis, A., & López-López, P. C. (2022). Gamification in ecology-oriented mobile applications: Typologies and purposes. Societies, 12(2), 42-53.

Saleem, A. N., Noori, N. M., & Ozdamli, F. (2022). Gamification applications in e-learning: A literature review. Technology, Knowledge and Learning, 27(1), 139-159.

Flavián, C., Ibáñez-Sánchez, S., Orús, C., & Barta, S. (2024). The dark side of the metaverse: The role of gamification in event virtualization. International Journal of Information Management, 75(23), 102726–102738.

Glikson, E., & Woolley, A. W. (2020). Human trust in artificial intelligence: A review of empirical research. Academy of Management Annals, 14(2), 627-660.

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8(1), 75264–75278.

Paul, D., Sanap, G., Shenoy, S., Kalyane, D., Kalia, K., & Tekade, R. K. (2021). Artificial intelligence in drug discovery and development. Drug Discovery Today, 26(1), 80-93.

Shi, F., Wang, J., Shi, J., Wu, Z., Wang, Q., Tang, Z., He, K., Shi, Y., & Shen, D. (2021). Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19. IEEE Reviews in Biomedical Engineering, 14(1), 4-15.

Zhang, C., & Lu, Y. (2021). Study on artificial intelligence: The state of the art and future prospects. Journal of Industrial Information Integration, 23(3), 100224-100238.

Okunaiya, O., Austin, R., & Zhu, S. Y. (2024). ChatGPT-enabled network automation using API-based prompts. In NOMS 2024-2024 IEEE Network Operations and Management Symposium (pp. 1-5). Seoul: IEEE.

Lappalainen, Y., & Narayanan, N. (2023). Aisha: A custom AI library chatbot using the ChatGPT API. Journal of Web Librarianship, 17(3), 37-58.

Chen, B. H., & Chen, C. C. (2023). Invention of Line-ChatBot: An innovative application of ChatGPT API and LINE Bot for enhanced student learning. In 2023 IEEE 6th International Conference on Knowledge Innovation and Invention (ICKII) (pp. 452-455). Sapporo: IEEE.

Abasi, A., Hoseinabadi, R., Raji, P., Friedman, J. H., & Hadian, M. R. (2022). Evaluating oculomotor tests before and after vestibular rehabilitation in patients with Parkinson's disease: A pilot pre-post study. Parkinson's Disease, 2022(1), 1-6.

Craig, S., Stark, P., Wilson, C. B., Carter, G., Clarke, S., & Mitchell, G. (2023). Evaluation of a dementia awareness game for undergraduate nursing students in Northern Ireland: A pre-/post-test study. BMC Nursing, 22(1), 177-1.

Al-Mughairi, H., & Bhaskar, P. (2024). Exploring the factors affecting the adoption of AI techniques in higher education: Insights from teachers' perspectives on ChatGPT. Journal of Research in Innovative Teaching & Learning, 17(1), 110-123.

Li, T., Ji, Y., & Zhan, Z. (2024). Expert or machine? Comparing the effects of pairing student teachers with in-service teachers and ChatGPT on their critical thinking, learning performance, and cognitive load in an integrated-STEM course. Asia Pacific Journal of Education, 44(1), 1-16.

Guo, Y., & Lee, D. (2023). Leveraging ChatGPT for enhancing critical thinking skills. Journal of Chemical Education, 100(12), 4876–4883.

Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y. S., Gašević, D., & Chen, G. (2023). Can large language models provide feedback to students? A case study on ChatGPT. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 323-325). Orem, UT, USA: IEEE.

Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students' writing. Learning and Instruction, 91(1), 1-15.

Abolnejadian, M., Alipour, S., & Taeb, K. (2024). Leveraging ChatGPT for adaptive learning through personalized prompt-based instruction: A CS1 education case study. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (pp. 1-8). New York, NY, USA: ACM.

Rusandi, M. A., Ahman, Saripah, I., Khairun, D. Y., & Mutmainnah. (2023). No worries with ChatGPT: Building bridges between artificial intelligence and education with critical thinking soft skills. Journal of Public Health (Bangkok), 45(3), e602–e603.