Evaluation of HOTS Test on Renewable Energy Materials through Rasch Model Analysis
DOI:
10.29303/jpm.v20i5.9458Published:
2025-07-30Issue:
Vol. 20 No. 5 (2025)Keywords:
HOTS; Ministep; Physics; Rasch ModelArticles
Downloads
How to Cite
Downloads
Metrics
Abstract
The growth of education in the globalisation era demands a paradigm change in the learning process, especially in terms of assessment that emphasises the importance of higher-order thinking skills (HOTS), which are no longer only focused on memorising and understanding basic concepts. This study aims to evaluate HOTS tests on renewable energy materials using the Rasch model as the main analysis technique. This research uses an evaluative design with a quantitative approach that aims to analyse the psychometric quality of HOTS test instruments objectively and systematically. The research subjects consisted of 71 learners selected through a purposive sampling technique from two classes in senior high school. The research instrument used was 10 multiple-choice questions with five answer options and analysed using Ministep software. The analysis results through the Wright map show that the distribution of students' abilities is close to a normal distribution. In contrast, the distribution of item difficulties has an uneven pattern, so that it does not cover the entire range of students' HOTS abilities. In addition, most of the items had adequate psychometric quality, but two items did not fit and needed special attention. Nevertheless, further analysis using the Item Characteristic Curve (ICC) revealed that item S3 indicated misfit, bringing the total to three items that did not fit the Rasch Model predictions. In addition, item S5 is included as a bias item. Overall, the instrument shows that the hierarchy of item difficulty is consistent and reliable, but it has limitations in sensitivity in distinguishing ability levels between learners. In addition, ICC analysis provides more sensitive detection of response pattern discrepancies, indicating the need for multiple analyses for comprehensive validation. Additional items are needed to comprehensively cover the spectrum of student abilities and improve the precision of identifying individual ability differences.
References
R. Febriana, Evaluasi Pembelajaran. Jakarta: PT Bumi Aksara, 2019.
M. Astuti, Evaluasi Pendidikan. Sleman: Penerbit Deepublish, 2022.
D. A. Adam, Khaeruddin, and K. Arafah, “Pengembangan Instrumen Tes Hasil Belajar Kognitif Fisika Kelas XI MIPA SMA Negeri 2 Majene,” J. Sains dan Pendidik. Fis., vol. 19, no. 2, pp. 183–193, 2023, [Online]. Available: https://ojs.unm.ac.id/JSdPF/article/view/34499/23335
N. Mappalesye, S. S. Sari, and K. Arafah, “Pengembangan Instrumen Tes Kemampuan Berpikir Kritis Dalam Pembelajaran Fisika,” J. Sains dan Pendidik. Fis., vol. 17, no. 1, pp. 69–82, 2021.
L. W. Anderson and D. R. Krathwohl, A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Addison Wesley Longman, 2001.
L. Ayumniyya and W. Setyarsih, “Profil Kemampuan Berpikir Tingkat Tinggi Siswa SMA dalam Pemecahan Masalah pada Materi Hukum Newton,” IPF Inov. Pendidik. Fis., vol. 10, no. 1, pp. 50–58, 2021, doi: 10.26740/ipf.v10n1.p50-58.
A. Saddia, Sutrisno, M. Saldi, and M. N. Agriawan, “Analisis Kemampuan Menyelesaikan Soal Hots Fisika Siswa Sma Di Kota Majene,” J. Fis. dan Pembelajarannya, vol. 4, no. 1, pp. 1–5, 2021, doi: 10.31605/phy.v4i1.1275.
V. Verdiana, H. Munawaroh, and F. Fatiatun, “Analisis Peningkatan Hasil Belajar Siswa pada Mata Pelajaran Fisika pada penerapan Model Pembelajaran PBL Menggunakan Soal HOTS,” Biochephy J. Sci. Educ., vol. 4, no. 1, pp. 70–74, 2024, doi: 10.52562/biochephy.v4i1.981.
Fitriana, S. S. Sitompul, and M. M. S. H., “Analisis Kemampuan Kognitif Peserta Didik dalam Menyelesaikan Soal HOTS Fisika Materi Getaran Harmonis,” J. Dunia Pendidik., vol. 5, no. 2, pp. 553–565, 2024, [Online]. Available: http://jurnal.stokbinaguna.ac.id/index.php/JURDIP/article/view/2083
B. Sumintono and W. Widhiarso, Aplikasi Model Rasch untuk Penelitian Ilmu-ilmu Sosial, Edisi Revi. Cimahi: Trim Komunikata Publishing House, 2014.
S. E. Embretson and S. P. Reise, Item Response Theory For Psychologists. New Jersey: Lawrence Erlbaum Associates, Inc, 2000.
K. S. Shultz, D. J. Whitney, and M. J. Zickar, Measurement Theory in Action Case Studies and Exercises, Third. New York: Routledge, 2021.
D. Saepuzaman, E. Istiyono, H. Haryanto, H. Retnawati, and Y. Yustiandi, “Analisis Karakteristik Butir Soal Fisika Dengan Pendekatan IRT Penskoran Dikotomus dan Politomus,” Radiasi J. Berk. Pendidik. Fis., vol. 14, no. 2, pp. 62–75, 2021, doi: 10.37729/radiasi.v14i2.1200.
R. M. Furr and V. R. Bacharach, Psychometrics An Introduction. California: Sage Publications, Inc, 2008.
N. L. A. Kassim, The Rasch Model for Psychometric Testing Rater-Mediated Assessment and Standard Setting. Kuala Lumpur: IIUM Press, 2024.
M. Wilson and K. Draney, “A Strategy for the Assessment of Competencies in Higher Education: The BEAR Assessment System,” in Modeling and Measuring Competencies in Higher Education, Rotterdam: Sense Publishers, 2013.
L. Kamilia, “Analisis Soal HOTS Materi Asam Basa untuk Mengukur Keterampilan Berpikir Kritis dengan Rasch Model,” Pentagon J. Mat. dan Ilmu Pengetah. Alam, vol. 3, no. 1, pp. 99–111, 2025, doi: https://doi.org/10.62383/pentagon.v3i1.411.
R. P. Yudha, “Higher Order Thinking Skills (HOTS) Test Instrument: Validity and Reliability Analysis with The Rasch Model,” EduMa Math. Educ. Learn. Teach., vol. 12, no. 1, pp. 21–38, 2023, doi: 10.24235/eduma.v12i1.9468.
R. Irmayanti, M. Rusdi, and Yusnaidar, “The Rasch Model: Implementation of Physics Learning Evaluation Instrument Based on Higher Order Thinking Skills,” Integr. Sci. Educ. J., vol. 4, no. 2, pp. 62–68, 2023, doi: 10.37251/isej.v4i2.325.
C. D. Anggraeni, M. Junus, and P. Damayanti, “Analisis Soal Latihan pada Buku Soal Fisika Kelas XI Berdasarkan Taksonomi Bloom Revisi Dilihat dari Prespektif Higher Order Thinking Skill pada Pokok Bahasan Fluida,” J. Literasi Pendidik. Fis., vol. 4, no. 1, pp. 40–51, 2023, doi: 10.30872/jlpf.v4i1.1945.
Sugiyono, Metode Penelitian Kuantitatif, Kualitatif, dan R&D. Bandung: CV. Alfabeta, 2017.
Riduwan, Dasar-dasar Statistika. Bandung: Alfabeta, 2015.
B. Sumintono and W. Widhiarso, Aplikasi Pemodelan Rasch Pada Assessment Pendidikan. Cimahi: Penerbit Trim Komunikata, 2015.
J. Irawan, S. Hadi, Z. Zulandri, J. Jamaluddin, A. Syukur, and S. Hadisaputra, "Validating metacognitive awareness inventory (MAI) in chemistry learning for senior high school: A rasch model analysis," J. Pijar MIPA, vol. 16, no. 4, pp. 442–448, 2021. [Online]. Available: https://doi.org/10.29303/jpm.v16i4.2603
S. E. Stemler and A. Naples, “Rasch Measurement v. Item Response Theory: Knowing When to Cross the Line,” Pract. Assessment, Res. Eval., vol. 26, pp. 1–16, 2021, doi: 10.7275/v2gd-4441.
S. P. Reise and D. A. Revicki, Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment. New York: Routledge, 2015.
Author Biographies
Nur Rahmatillah, Department of Physics, Universitas Negeri Surabaya
Abu Zainuddin, Department of Physics, Universitas Negeri Surabaya
License
Copyright (c) 2025 Nur Rahmatillah, Abu Zainuddin
This work is licensed under a Creative Commons Attribution 4.0 International License.
The following terms apply to authors who publish in this journal:
1. Authors retain copyright and grant the journal first publication rights, with the work simultaneously licensed under a Creative Commons Attribution License 4.0 International License (CC-BY License) that allows others to share the work with an acknowledgment of the work's authorship and first publication in this journal.
2. Authors may enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., posting it to an institutional repository or publishing it in a book), acknowledging its initial publication in this journal.
3. Before and during the submission process, authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website), as this can lead to productive exchanges as well as earlier and greater citation of published work (See The Effect of Open Access).
