Development and Initial Psychometric Evaluation of a Web-Based Rubric for Assessing Creativity in Mathematics Learning Media Design
DOI:
https://doi.org/10.48161/qaj.v6n1a2278Keywords:
rubric-based assessment; formative assessment; pre-service teacher education; higher education Indonesia, in-strument development.Abstract
This study aimed to develop and conduct an initial psychometric evaluation of GoSkill, a web-based assessment instrument specifically designed to measure student creativity within Mathematics Learning Media courses. This innovation addresses the growing necessity for technology-enhanced tools that support systematic and objective evaluation of creative performance in higher education. Employing a Research and Development (R&D) framework, the research encompassed stages of needs analysis, instrument design, product development, expert validation, limited field testing and reliability analysis. The study involved 84 mathematics education students enrolled in the Mathematics Learning Media course at four universities in East Java, Indonesia: University of PGRI Jombang, KH. A. Wahab Hasbullah University, Islamic University of Majapahit, and Wahidiyah University. Content validity, assessed by four experts in instructional technology, yielded a Scale-level Content Validity Index (S-CVI/Ave) of 0.89 indicating good content validity. Construct validity was further examined through Exploratory Factor Analysis (EFA), which confirmed a unidimensional structure with a single dominant factor explaining 53.45% of the total variance (KMO = 0.76). Item factor loadings ranged from 0.62 to 0.81, while reliability analysis demonstrated high internal consistency (Cronbach’s alpha = 0.865; McDonald’s omega = 0.87). These findings suggest that GoSkill was a feasible instrument for the formative assessment of student creativity in designing mathematics learning media. However, as this study represents an initial validation, further research utilizing advanced analytical techniques and broader, more diverse samples is required to establish more comprehensive psychometric evidence for the instrument.
Downloads
References
Kemendiktisaintek. (2025). Peraturan Menteri Pendidikan Tinggi, Sains, dan Teknologi Republik Indonesia Nomor 39 Tahun 2025 tentang penjaminan mutu pendidikan tinggi. Indonesia: Kemendiktisaintek.
Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. John Wiley & Sons.
Torrance, E. P. (1966). Torrance tests of creative thinking. Scholastic Testing Service.
Silver, E. A. (1997). Fostering creativity through instruction rich in mathematical problem solving and problem posing. ZDM, 29(3), 75–80.
Guilford, J. P. (1975). Creativity: A quarter century of progress. Journal of Creative Behavior, 9(2), 75–88.
Siswono, T. Y. E. (2016). Proses berpikir kreatif siswa dalam memecahkan dan mengajukan masalah matematika. Jurnal Ilmu Pendidikan, 15(1), 60–68.
Chang, C. C., Liang, C., Chou, P. N., & Lin, G. Y. (2017). Is game-based learning better in flow experience and various types of cognitive load than non-game-based learning? Computers in Human Behavior, 71, 218–227.
Hidayati, W., Iffah, J., Tristanti, L., & Nabilah, A. (2024). Digital assessment model to identify student creativity in constructing mathematics instructional media. Journal of Hunan University Natural Sciences, 51(9).
Shaban, A. A., & Ali, R. I. (2026). Toward Human-Centered Artificial Intelligence in Education: Adaptive Learning Models for Personalized and Equitable Academic Outcomes. Qubahan Techno Journal, 5(1), 1-16.
Andriani, R., Mellyzar, M., Lukman, I. R., Muttakin, M., Pasaribu, A. I., & Fadli, M. R. (2023). A review of digital assessment in education: Tools, features, and effectiveness. In Proceedings of the Malikussaleh International Conference on Education, Social Humanities, and Innovation (MICESHI).
Barbot, B., Besançon, M., & Lubart, T. (2015). Creative potential in educational settings: Its nature, measure, and nurture. Education, 43(4), 371–381.
OECD. (2023). PISA 2022 assessment and analytical framework. OECD Publishing.
Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences. Journal of Curriculum Studies, 44(3), 299–321.
Huang, J., Saleh, S., & Liu, Y. (2021). A review on artificial intelligence in education. Academic Journal of Interdisciplinary Studies, 10(3).
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., et al. (2023). ChatGPT for good? Opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274.
Said-Metwaly, S., Van den Noortgate, W., & Kyndt, E. (2017). Approaches to measuring creativity: A systematic literature review. Creativity Research Applications, 4(2), 238–275.
Beaty, R. E., Kenett, Y. N., Christensen, A. P., Rosenberg, M. D., Benedek, M., Chen, Q., et al. (2021). Robust prediction of individual creative ability from brain functional connectivity. Proceedings of the National Academy of Sciences, 118(8), e2022346118.
Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. ASCD.
Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, validity, and factor structure of the creative achievement questionnaire. Creativity Research Journal, 17(1), 37–50.
Osborn, A. F. (1953). Applied imagination: Principles and procedures of creative problem solving. Scribner.
Redecker, C., & Johannessen, Ø. (2013). Changing assessment—Towards a new assessment paradigm using ICT. European Journal of Education, 48(1), 79–96.
Shute, V. J., & Rahimi, S. (2017). Review of computer-based assessment for learning. Journal of Computer Assisted Learning, 33(1), 1–19.
Bennett, R. E. (2015). The changing nature of educational assessment. Review of Research in Education, 39(1), 370–407.
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2002). On the roles of task model variables in assessment design. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 97–128). Routledge.
Besemer, S. P., & Treffinger, D. J. (1981). Analysis of creative products: Review and synthesis. Journal of Creative Behavior, 15(3), 158–178.
Mardapi, D. (2012). Pengukuran penilaian dan evaluasi pendidikan. Nuha Medika.
Messick, S. (1995). Validity of psychological assessment. American Psychologist, 50(9), 741–749.
Arslan, A. (2020). Reliability and validity of instruments measuring English teachers’ TPACK. International Journal of Assessment Tools in Education, 7(3), 343–360.
Petra, T. Z. H. T., & Aziz, A. M. J. (2020). Investigating reliability and validity of student performance assessment using Rasch model. Journal of Physics: Conference Series, 1529(4), 042088.
Azwar, S. (2022). Reliabilitas dan validitas (4th ed.). Pustaka Pelajar.
El-Hamamsy, L., Zapata-Cáceres, M., Barroso, E. M., Mondada, F., Zufferey, J. D., & Bruno, B. (2022). The competent computational thinking test. Journal of Educational Computing Research, 60(7), 1818–1866.
Akib, E., & Ghafar, M. N. A. (2015). The validity and reliability of assessment for learning. Education Journal, 4(2), 64–68.
Jin, Y., Martinez-Maldonado, R., Gašević, D., & Yan, L. (2025). GLAT: The generative AI literacy assessment test. Computers and Education: Artificial Intelligence, 100436.
Gaggioli, A., Casaburi, G., Ercolani, L., Collovà, F., Torre, P., & Davide, F. (2025). Assessing reliability and validity of large language models for essay assessment. arXiv preprint.
Kullan, S., Mansor, M., & Ishak, R. (2022). Validity and reliability of learning organization instruments. International Journal of Evaluation and Research in Education, 11(4), 1725–1733.
Taguma, M., Feron, E., & Lim, M. H. (2018). Future of education and skills 2030. OECD.
Cimatti, B. (2016). Definition and assessment of soft skills. International Journal for Quality Research, 10(1), 97.
Heckman, J. J., & Kautz, T. (2012). Hard evidence on soft skills. Labour Economics, 19(4), 451–464.
Succi, C., & Canovi, M. (2020). Soft skills and employability. Studies in Higher Education, 45(9), 1834–1847.
Escolà-Gascón, Á., & Gallifa, J. (2022). Measuring soft skills in education. Studies in Educational Evaluation, 74, 101155.
Colledani, D., Robusto, E., & Anselmi, P. (2024). Assessing key soft skills. Frontiers in Psychology, 15, 1405822.
Phuti, F., Koloi-Keaikitse, S., Tsheko, G. N., & Oppong, S. (2023). Soft skills assessment scale. SAGE Open, 13(4).
Al-Sa’di, A., Yamjal, P., Ahmad, E., Panjabi, R., Allott McPhee, C. A. M., & Guler, O. (2023). Assessing educators’ soft skills. Administrative Sciences, 13(9), 208.
De Fruyt, F., & Scheirlinckx, J. (2024). Assessment of social-emotional skills. ECNU Review of Education.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge. Teachers College Record, 108(6), 1017–1054.
Torrance, E. P. (1974). Torrance tests of creative thinking. Personnel Press.
Leikin, R. (2006). Mathematical connections and problem solving. Aleh Mathematics Journal, 36, 8–14.
Beghetto, R. A., & Kaufman, J. C. (2014). Classroom contexts for creativity. High Ability Studies, 25(1), 53–69.
Koehler, M., & Mishra, P. (2009). What is TPACK? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70.
Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., & Howe, A. (2013). Creative learning environments. Thinking Skills and Creativity, 8, 80–91.
Henriksen, D., Mishra, P., & Fisser, P. (2016). Creativity and technology in education. Journal of Educational Technology & Society, 19(3), 27–37.
Sawyer, R. K., & Henriksen, D. (2024). Explaining creativity: The science of human innovation. Oxford University Press.
Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004). Why isn’t creativity more important? Educational Psychologist, 39(2), 83–96.
Leikin, R., & Lev, M. (2013). Mathematical creativity differences. ZDM, 45(2), 183–197.
Wahyu, M. N., Sutiarso, S., & Bharata, H. (2020). Soft skills communication learning. Jurnal Cendekia, 4(1), 406–413.
Sattriawan, A., Sutiarso, S., & Rosidin, U. (2020). Interactive learning media. Jurnal Cendekia, 4(2), 950–963.
Mustofa, G. G. (2021). Soft skills assessment using e-rubric (Thesis). Universitas Pendidikan Indonesia.
Husairi, H., & Sodikin, S. (2023). Soft skills and AI in education. Al-Jadwa Journal, 3(1), 111–138.
Gough, H. G. (1979). The creative personality scale. Consulting Psychologists Press.
Munandar, U. (2016). Pengembangan kreativitas anak berbakat. Rineka Cipta.
Guo, P., Saab, N., Post, L. S., & Admiraal, W. (2020). Project-based learning review. International Journal of Educational Research, 102, 1–13.
Dampierre, M. R. de, & Gaya-López, P. J. L. B. (2024). Project-based learning in engineering. Education Sciences, 14.
Panadero, E., Ortube, A. F., Krebs, R., & Roelle, J. (2025). Online rubric platforms. Assessment & Evaluation in Higher Education, 50, 31–49.
Panadero, E., Jonsson, A., Pinedo, L., & Fernández-Castilla, B. (2023). Effects of rubrics. Educational Psychology Review, 35.
Cheng, Y. P., & Huang, Y. M. (2025). LUPDA rubric model. International Journal of STEM Education, 45, 1–31.
Mulet, E., Royo, M., Chulvi, V., & Galán, J. (2017). Creativity and design outcomes. Dyna, 84(200), 38–45.
Han, J., Forbes, H., & Schaefer, D. (2021). Creativity and design relationships. Research in Engineering Design, 32(3), 289–307.
Muneer, S., Santhosh, M., Parangusan, H., & Bhadra, J. (2025). Design thinking meta-analysis. International Journal of Technology and Design Education, 1–31.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis.
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55.
Hidayati, W. S., & Tristanti, L. B. (2023). Creativity profile in mathematics learning. JTAM, 7, 836–847.
Windyariani, S., & Setiono, S. (2024). Creative thinking rubric in STEM. Journal of Innovative Science Education, 13(50), 64–73.
Alruwais, N., Wills, G., & Wald, M. (2018). Advantages and challenges of e-assessment. International Journal of Information and Education Technology, 8(1), 34–37.
Ibarra-Sáiz, M. S., Gómez-Ruiz, M. Á., Balderas, A., & Rodríguez-Gómez, G. (2025). Technology-enhanced assessment. Technology, Knowledge and Learning, 1–31.
Trujillo, B. P. S., Velarde-Camaqui, D., Gonzales Nuñez, C. A., Castillo Silva, E. V., & Gonzalez Said de la Oliva, M. D. P. (2025). Formative assessment review. Frontiers in Education, 10, 1509983.
Nicol, D., Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning. Studies in Higher Education, 31, 199–218.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–74.
Kingston, N., & Nash, B. (2011). Formative assessment meta-analysis. Educational Measurement: Issues and Practice, 30(4), 28–37.
Wiliam, D. (2018). Assessment for learning implementation. Assessment in Education, 25(6), 682–685.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Qubahan Academic Journal
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
