Integration of Augmented Reality and Motion Capture Technology in the Development of Interactive Animation Based on the Digital Story of Malin Kundang

Joko Sutopo(1), Anggita Taalungan(2), Charles De Haan(3), RB Hendri Kuswantoro(4), Abdul Syukor Mohamad Jaya(5),

DOI: https://doi.org/10.46843/jpm.v4i1.383


(1) Faculty of Science and Technology, Universitas Teknologi Yogyakarta, Indonesia
(2) Faculty of Science and Technology, Universitas Teknologi Yogyakarta, Indonesia
(3) Animation and Game, Sekolah Tinggi Multimedia MMTC Yogyakarta, Indonesia
(4) Animation and Game, Sekolah Tinggi Multimedia MMTC Yogyakarta, Indonesia
(5) Faculty of Information and Communication Technology, Universiti Teknikal Malaysia Melaka, Malaysia

Abstract

The folk tale of Malin Kundang carries significant moral and cultural values, yet younger generations' interest in conventional storytelling has declined with the advancement of digital technologies. This research addresses that gap by developing an interactive digital animation using Augmented Reality (AR) and Motion Capture (MoCap) technologies to create a more immersive and engaging user experience. Through a mixed-methods approach, including prototype development with Unity 3D, motion tracking using OptiTrack/Xsens, and user testing, the system achieved 95% accuracy in character motion and high user satisfaction. While technical challenges such as latency and hardware constraints remain, the results demonstrate that the integration of AR and MoCap revitalizes traditional stories and offers global benefits—providing scalable solutions applicable to education, cultural heritage preservation, training simulations, and entertainment industries worldwide. This study contributes to the growing field of immersive digital storytelling and offers insights for future research into motion algorithm optimization and developing more accessible MoCap technologies for global use.

Keywords

Interactive animation; augmented reality; digital storytelling; digital legacy; motion capture

Full Text:

PDF

References

Abdulghani, T., & Sembada, R. M. (2021). Pemanfaatan Teknologi Augmented Reality untuk Memilih Model Kacamata di Central Optikal 165 dengan Menggunakan Metode Markerless Berbasis Android. Media Jurnal Informatika, 13(1), 36–44.

Alfian, M., Dwi Putra, A., & Surahman, A. (2022). Penerapan Augmented Reality (Tanaman Obat Keluarga) Toga Sebagai Media Pembelajaran Berbasis Android Dengan Metode Marker. Jurnal Informatika Dan Rekayasa Perangkat Lunak (JATIKA), 3(1), 77–85.

Anggara, R. P., Musa, P., Lestari, S., & Widodo, S. (2021). Application of Electronic Learning by Utilizing Virtual Reality (VR) and Augmented Reality (AR) Methods in Natural Sciences Subjects (IPA) in Elementary School Students Grade 3. Jurnal Teknologi Pendidikan, 23(1), 58–69.

Budhi, A. S., Logayah, D. S., Darmawan, R. A., & Tianti, T. (2022). Pengembangan Model Augmented Reality (DISMUS) sebagai Media Pembelajaran Digital IPS. Heritage: Journal of Social Studies |, 3(2), 176–190.

Cannavò, A., Bottino, F., & Lamberti, F. (2024). Supporting Motion-capture Acting with Collaborative Mixed Reality. Computers and Graphics (Pergamon), 124, 1–10.

Devagiri, J. S., Paheding, S., Niyaz, Q., Yang, X., & Smith, S. (2022). Augmented Reality and Artificial Intelligence in industry: Trends, Tools, and Future Challenges. Expert Systems with Applications, 207, 118002.

El-Raheb, K., Kougioumtzian, L., Kalampratsidou, V., Theodoropoulos, A., Kyriakoulakos, P., & Vosinakis, S. (2025). Sensing the Inside Out: An Embodied Perspective on Digital Animation Through Motion Capture and Wearables. Sensors, 25(7), 1–23.

Habeeb, U., Rahaman, S., Ahamed Rasool S, M., Monteza, A. M., & Hermansyah, S. (2025). Evaluating RADiCAL Motion Capture AI: A study on Pre-Visualization Workflows. Jurnal Teknologi Pendidikan, 3(1), 41–50.

Hadi, F., & Permana, R. (2021). Augmented Reality Point Of Interest (POI) Pariwisata Kota Padang Berbasis Android. Indonesian Journal of Computer Science Attribution-ShareAlike, 4(1), 41.

Iqbal, J., & Sidhu, M. S. (2022). Acceptance of Dance Training System Based on Augmented Reality and Technology Acceptance Model (TAM). Virtual Reality, 26(1), 33–54.

Jalaludin, M. (2024). Media Pembelajaran Interaktif Pengenalan Ilmu Tajwid Berbasis Augmented Reality. Jurnal Ilmiah Informatika, 12(2), 156–162.

Jin, Y., Ma, M., & Zhu, Y. (2022). A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality. Multimedia Tools and Applications, 81(4), 5795–5826.

Khatib, J. S., & Citrawati, A. (2024). Seni Tari di Era Virtual dan Augmented Reality. Indonesian Journal of Computer Science, 13(2), 3029–3037.

Maulidannisa, N., & Ansori, I. (2024). Development of Augmented Reality Media on PjBL Learning Model of Human Motion Organ Material to Improve Learning Outcomes. Jurnal Pijar Mipa, 19(3), 459–463.

Nafisatun, N. M., Lestari, F. P., Anggreini, C. A., Yasinta, E., & Ubaidah, N. (2024). Kontinu: Jurnal Penelitian Didaktik Matematika Pull Out Photo Box with AR Technology in Supporting Mathematics Literacy in Deaf Crew Members. Kontinu: Jurnal Penelitian Didaktik Matematika, 8(2), 135–151.

Novia, L., & Zalilludin, D. (2021). Aplikasi Media Pembelajaran Mengenal Alat Musik Tradisional Untuk Anak-Anak Berbasis Augmented Reality Pada Perangkat Mobile. Jurnal IKRA-ITH Informatika, 5(1), 15–21.

Nurrisma., Munadi, R., Syahrial, & Meutia, E. D. (2021). Perancangan Augmented Reality dengan Metode Marker Card Detection dalam Pengenalan Karakter Korea. Informatika Mulawarman : Jurnal Ilmiah Ilmu Komputer, 16(1), 34–41.

Palmarini, R., Del Amo, I. F., Ariansyah, D., Khan, S., Erkoyuncu, J. A., & Roy, R. (2023). Fast Augmented Reality Authoring: Fast Creation of AR Step-by-Step Procedures for Maintenance Operations. IEEE Access, 11, 8407–8421.

Pieta, P., Jegierski, H., Babiuch, P., Jegierski, M., Plaza, M., Lukawski, G., Deniziak, S., Jasinski, A., Opalka, J., Wegrzyn, P., Igras-Cybulska, M., & Lapczynski, A. (2024). Automated Classification of Virtual Reality User Motions Using a Motion Atlas and Machine Learning Approach. IEEE Access, 12, 94584–94609.

Rauschnabel, P. A., Felix, R., Hinsch, C., Shahab, H., & Alt, F. (2022). What is XR? Towards a Framework for Augmented and Virtual Reality. Computers in Human Behavior, 133, 1–18.

Salim, A. R., Ibrahim, N., & Muslim, S. (2020). Developing Flipbook Multimedia For Learning Motion Photos Techniques For Vocational High School Grade XII. International Journal Of Education, Information Technology And Others (IJEIT), 3, 551–555.

Santi, G. M., Ceruti, A., Liverani, A., & Osti, F. (2021). Augmented Reality in Industry 4.0 and Future Innovation Programs. Technologies, 9(2).

Santiyasaa, I. W., Narotamaa, I. G. A., Suhartanaa, I. K. G., Putraa, I. G. N. A. C., Mahendraa, I. B. M., & Widiartha, I. M. (2022). Perancangan Aplikasi Augmented Reality Location Based Service Nusa Dua Tourism Guide. Jurnal Elektronik Ilmu Komputer Udayana , 11(2), 235–246.

Santoso, R. A., & Cahyono, N. (2024). Implementation of Augmented Reality 3D Catalog And 2D Motion Based on Multimarker. Jurnal Teknologi Informasi Dan Pendidikan, 17(1), 107–121.

Scuttari, A. (2021). Tourism experiences in motion. Mobile, visual and psychophysiological methods to capture tourists "on the move." Tourism Management Perspectives, 38, 100825.

Singh, G., & Ahmad, F. (2024). An interactive augmented reality framework to enhance the user experience and operational skills in electronics laboratories. Smart Learning Environments, 11(1).

Suriawan, M. V., Fabroyir, H., & Herumurti, D. (2025). Evaluating Object Collection In Emergency Simulations Using Virtual And Augmented Reality. JUTI: Jurnal Ilmiah Teknologi Informasi, 23(1), 65–80.

Syafiih, M., Rojabi, A. R., & Firdaus, Moch. R. (2023). Strategi Peningkatan Daya Saing Wisata Tirta Agung Wonosari Bondowoso Melalui Transformasi Digital, Teknologi Augmented Reality, dan Sosial Media Marketing. Jurnal Kecerdasan Buatan, Komputasi Dan Teknologi Informasi, 4(2), 158–164.

Syed, T. A., Siddiqui, M. S., Abdullah, H. B., Jan, S., Namoun, A., Alzahrani, A., Nadeem, A., & Alkhodre, A. B. (2023). In-Depth Review of Augmented Reality: Tracking Technologies, Development Tools, AR Displays, Collaborative AR, and Security Concerns. Sensors, 23(1).

Tiwari, A. S., Bhagat, K. K., & Lampropoulos, G. (2024). Designing and evaluating an augmented reality system for an engineering drawing course. Smart Learning Environments, 11(1), 1–19.

Wibowo, M. C., Nugroho, S., & Wibowo, A. (2024). The Use of Motion Capture Technology in 3D animation. International Journal of Computing and Digital Systems, 15(1), 975–987.

Refbacks

  • There are currently no refbacks.