Evaluating Complex Problem-Solving Test Instrument Quality in Measuring Computational Thinking Skills in Global Warming: Rasch Model Analysis

Farika Nurfalah(1), Judhistira Aria Utama(2), Muslim Muslim(3),

DOI: https://doi.org/10.46843/jpm.v4i2.425


(1) Department of Master of Physics Education, Faculty of Mathematics and Natural Sciences, Universitas Pendidikan Indonesia, Indonesia
(2) Department of Master of Physics Education, Faculty of Mathematics and Natural Sciences, Universitas Pendidikan Indonesia, Indonesia
(3) Department of Master of Physics Education, Faculty of Mathematics and Natural Sciences, Universitas Pendidikan Indonesia, Indonesia

Abstract

This research aims to develop and evaluate a high-quality Complex Problem Solving (CPS) test instrument based on global warming issues to assess students' Computational Thinking (CT) skills. Global warming was chosen because of its contextual relevance in 21st-century education and its potential to promote higher-order thinking. The instrument consists of six essay questions that require students to demonstrate complex problem-solving skills, including decomposition, pattern recognition, abstraction, and algorithmic thinking. The method used in this study is Research and Development (R&D), which contains five stages: (1) analysis of needs, (2) development of the instrument, (3) validation by an expert, (4) trial instruments, and (5) data analysis using the Rasch Model. This instrument was evaluated using the Rasch model to examine the validity, reliability, and difficulty level of its items, which were analyzed using the Minister 3.80.1 software. The results showed that the instrument achieved the "very valid" categories for content and construct validity. The raw variance explained by the measure was 76.20%, classified as "specific," indicating strong empirical validity. The Cronbach's alpha (KR-20) value was 0.91, indicating the instrument's high reliability. The item reliability value was 0.94, and the person reliability was 0.89. This means that students' consistency in answering questions was good, and the instrument had high reliability. The distribution of item difficulty levels was balanced, allowing students to be distinguished by their abilities at various levels. It can be concluded that this instrument is reliable and valid. Thus, this instrument can measure students' computational thinking skills on global warming.

Keywords

Complex Problem Solving (CPS), Computational Thinking (CT), Global Warming, Rasch Model, Instruments Test

Full Text:

PDF

References

Arifa, M. F. (2023). PBL-Based PhET Simulations E-Worksheet: Improvement Students' Communication Skill on Global Warming in Merdeka Curriculum. ACM International Conference Proceeding Series, 172–177.

Blue, J. (2023). Addressing Climate Change Challenges Through Multidisciplinary Collaboration. Journal of Environmental Policy, 2, 9–12.

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (3rd ed.). Routledge.

Boone, W. J. (2016). Rasch Analysis for Instrument Development: Why, When, and How? CBE Life Sciences Education, 15(4).

Christensen, D. (2023). Computational Thinking to Learn Environmental Sustainability: A Learning Progression. Journal of Science Education and Technology, 32(1), 26–44.

Cronbach, L. J., & Meehl, P. E. (1955). Construct Validity in Psychological Tests. Psychological Bulletin, 52(4), 281–302.

Engelhard Jr, G., & Wang, J. (2025). Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences. Routledge.

Fadilah, N. (2024). Analisis Kemampuan Computational Thinking Siswa SMP dalam Menyelesaikan Soal HOTS Ditinjau dari Tingkat Disposisi Matematis [Tesis, Universitas Pendidikan Indonesia].

Fajri, M., Yurniawati, & Utomo, E. (2019). Computational Thinking, Mathematical Thinking Berorientasi Gaya Kognitif pada Pembelajaran Matematika di Sekolah Dasar. Dinamika Sekolah Dasar, 1(1), 1–18.

Greiff, S., Fischer, A., & Stadler, M. (2015). Assessing Complex Problem-Solving Skills with Multiple Complex Systems. Thinking & Reasoning, 21(3), 356–382.

Hamzah, F. M., Rashid, M. N. A., Rahman, M. N. A., & Rasul, M. S. (2022). Evaluating the Validity and Reliability of Authentic Learning Instruments Using RASCH Model. International Journal of Global Optimization and Its Application, 1(3), 182–189.

IPCC. (2007). Report from Intergovernmental Panel on Climate Change.

Khine, M. S. (2020). Rasch Measurement: Applications in Quantitative Educational Research. Springer.

Kurnianto, A., Sunjaya, D. K., Rinawan, F. R., & Hilmanto, D. (2023). Validation and Reliability of Lifestyle Instruments for Indonesian Adolescents with Hypertension Family History: A Rasch Model. Open Access Macedonian Journal of Medical Sciences, 11(B), 109–114.

Ling, L. U., Tammie, C. S., Nasrah, N., Jane, L., & Norazila, A. A. (2018). An Evaluation Tool to Measure Computational Thinking Skills: Pilot Investigation. Herald NAMSA, 1, 606–614.

Maharani, S., Kholid, M. N., Pradana, L. N., & Nusantara, T. (2019). Problem Solving in the Context of Computational Thinking. Infinity: Journal of Mathematics Education, 8(2), 109–116.

Mardapi, D. (2008). Teknik Penyusunan Instrumen Tes dan Non Tes. Mitra Cendekia Press.

Nasution, I. B. (2019). Effectiveness Problem-Based Learning (PBL) with Reading Infusion Strategic to Improving Scientific Literacy for High School Students on Topic Global Warming. Journal of Physics: Conference Series, 1280(5).

Nouri, J., Zhang, L., Mannila, L., & Norén, E. (2020). Development of Computational Thinking, Digital Competence and 21st Century Skills When Learning Programming in K-9. Education Inquiry, 11(1), 1–17.

OECD. (2023). PISA 2022 Technical Report: 6. Sample Design.

Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch Analysis in Physics Education Research: Why Measurement Matters. Physical Review Physics Education Research, 15(2), 020111.

Rahmawati, E. N. (2020). Development of E-Handout Assisted by PhET Simulation with Problem Based Learning (PBL) Model About Momentum Conservation Law and Collision to Train Students' Conceptual Understanding. Journal of Physics: Conference Series, 1440(1).

Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating Computational Thinking with K-12 Science Education Using Agent-Based Computation: A Theoretical Framework. Education and Information Technologies, 18(2), 351–380.

Sugiyono. (2019). Metode Penelitian Kuantitatif, Kualitatif dan R&D. Alfabeta.

Sulsilah, H., Hidayat, A., & Ramalis, T. R. (2023). Analysis of Computational Thinking Instrument for High School Student Using Rasch Model. Jurnal Penelitian Pendidikan IPA, 9(3), 1445–1450.

Sumintono, B., & Widhiarso, W. (2014). Aplikasi Model Rasch untuk Penelitian Ilmu-Ilmu Sosial. Trim Komunikata.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi Pemodelan Rasch pada Assessment Pendidikan.

Supari., Tangang, F., Juneng, L., & Aldrian, E. (2017). Observed Changes in Extreme Temperature and Precipitation Over Indonesia. International Journal of Climatology, 37(4), 1979–1997.

Wahyudin, D., Subkhan, E., Malik, A., Hakim, M. A., Sudiapermana, E., Alhapip, M. L., Amalia, L. S., Ali, N. B. V., & Krisna, F. N. (2024). Kajian Akademik Kurikulum Merdeka. Kemendikbud.

Wijnia, L., Noordzij, G., Arends, L. R., Rikers, R. M. J. P., & Loyens, S. M. M. (2024). The Effects of Problem-Based, Project-Based, and Case-Based Learning on Students' Motivation: A Meta-Analysis. Educational Psychology Review, 36(1).

Wing, J. M. (2006). Computational Thinking. Communications of the ACM, 49(3), 33–35.

Yadav, A., Good, J., Voogt, J., & Fisser, P. (2017). Computational Thinking as an Emerging Competence Domain. In Technical and Vocational Education and Training (pp. 1051–1067). Springer.

Yin, Y., Hadad, R., Tang, X., & Lin, Q. (2020). Improving and Assessing Computational Thinking in Maker Activities: The Integration with Physics and Engineering Learning. Journal of Science Education and Technology, 29(2), 189–214.

Refbacks

  • There are currently no refbacks.