Deteksi Pose Semaphore Berbasis Deep Learning Menggunakan Metode Multi-Layer Perceptron dan MediaPipe
DOI:
https://doi.org/10.55606/juisik.v5i2.1386Keywords:
Confusion Matrix, MediaPipe, Multi-Layer Perceptron, Pose Detection, SemaphoreAbstract
Semaphore is a communication method used to send and receive messages by utilizing flags, paddles, sticks, bare hands, or gloves. Semaphore signaling is commonly used in scouting activities (Pramuka). However, novice scouts often face difficulties in accurately recognizing semaphore gestures. Although guidebooks are available, errors may still occur, especially when learning independently. To address this issue, a gesture detection system using the Multi-Layer Perceptron (MLP) method and MediaPipe framework is proposed, which aims to evaluate the accuracy of MLP and MediaPipe in detecting semaphore poses. To assess the performance of the MLP model, a Confusion Matrix is used to analyze the number of correct and incorrect predictions and generate metrics such as accuracy, precision, recall, and F1-score. The evaluation results show that the system successfully recognized 122 poses correctly, achieving an accuracy of 93.84%, precision of 93.84%, recall of 100%, and an F1-score of 96.80%. Based on these results, the system is considered sufficiently accurate and can be used as an interactive learning tool to recognize semaphore codes, particularly in scouting activities.
References
Alfarezi, M. (2024). Pengenalan kode semaphore berbasis Raspberry Pi menggunakan metode CNN. E-Proceeding of Engineering, 11(4), 4851–4858. https://doi.org/Diakses dari repository Universitas Telkom
Alfarizi, M. R. S., Al-farish, M. Z., Taufiqurrahman, M., Ardiansah, G., & Elgar, M. (2023). Penggunaan Python sebagai bahasa pemrograman untuk machine learning dan deep learning. Karya Ilmiah Mahasiswa Bertauhid (KARIMAH TAUHID), 2(1), 1–6.
Cai, S., Shu, Y., Chen, G., Ooi, B. C., Wang, W., & Zhang, M. (2019). Effective and efficient dropout for deep convolutional neural networks. arXiv, 1–12. https://doi.org/10.48550/arXiv.1904.03392
Dudekula, K. V., Chalapathi, M. M. V., Kumar, Y. V. P., Prakash, K. P., Reddy, C. P., Gangishetty, D., Solanki, M., & Singhu, R. (2024). Physiotherapy assistance for patients using human pose estimation with Raspberry Pi. ASEAN Journal of Scientific and Technological Reports, 27(4). https://doi.org/10.55164/ajstr.v27i4.251096
Dwika, A. S. P., Putra, A. S., & Alkadri, S. P. A. A. (2024). Identifikasi gerakan tangan pada sandi semaphore Pramuka secara realtime menggunakan decision tree. Jurnal Education and Teknologi, 5(2), 401–417.
Gulo, S. H., & Lubis, A. H. (2024). Penerapan multi-layer perceptron untuk mengklasifikasi penduduk kurang mampu. Journal of Computer Science and Information Technology, 4(2), 51–59. https://doi.org/10.47065/explorer.v4i2.1146
Handayanto, R. T., & Herlawati, H. (2020). Prediksi kelas jamak dengan deep learning berbasis graphics processing units. Jurnal Kajian Ilmiah, 20(1), 67–76. https://doi.org/10.31599/jki.v20i1.71
Mahersatillah Suradi, A., Alam, S., Rasyid, M. F., Djafar, I., Dipa Makassar, U., & Perintis Kemerdekaan, J. K. (2023). Sistem deteksi kantuk pengemudi mobil berdasarkan analisis rasio mata menggunakan computer vision. JUKI: Jurnal Komputer dan Informatika, 2, 222–230.
Mutianniza, S. Z., & Suwardoyo, U. (2023). Aplikasi kamera cerdas untuk deteksi kendaraan menggunakan library TensorFlow. Jurnal Sintaks Logika, 3(3), 61–68. https://doi.org/10.31850/jsilog.v3i3.2589
Ojiako, K., & Farrahi, K. (2023). MLPs are all you need for human activity recognition. Applied Sciences (Switzerland), 13(20). https://doi.org/10.3390/app132011154
Peryanto, A., Yudhana, A., & Umar, R. (2020). Rancang bangun klasifikasi citra dengan teknologi deep learning berbasis metode convolutional neural network. FORMAT: Jurnal Ilmiah Teknik Informatika, 8(2), 138. https://doi.org/10.22441/format.2019.v8.i2.007
Przybyła-Kasperek, M., & Marfo, K. F. (2024). A multi-layer perceptron neural network for varied conditional attributes in tabular dispersed data. PLoS ONE, 19(12). https://doi.org/10.1371/journal.pone.0311041
Putra, L. S. A., Wigyarinto, F. T. P., Kusumawardhani, E., & Gunawan, V. A. (2024). Semaphore letter code recognition system using wavelet method and back propagation neural network. International Journal of Advanced Technology and Engineering Exploration, 11(112), 316–331. https://doi.org/10.19101/IJATEE.2023.10102433
Ragil, K., Dyansyah, K., Purwantoro, S. D., & Ilmi, M. (2025). Penggunaan computer vision untuk estimasi pose squat sebagai solusi alternatif latihan kebugaran di gym. Seminar Nasional Teknologi & Sains (STAINS), 4, 199–207.
Resha, M., Syamsu, S., Andryanto, & Bakry. (2022). Prediksi penyebaran kasus tuberkulosis dengan metode artificial neural network dan multi-layer perceptron di Kota Makassar. JNSTA ADPERTISI Journal, 2(1), 16–21.
Sarakon, S., Massagram, W., & Tamee, K. (2025). Multisource data fusion using MLP for human activity recognition. Computers, Materials and Continua, 82(2), 2109–2136. https://doi.org/10.32604/cmc.2025.058906
Sejati, R. H. P., & Mardhiyyah, R. (2021). Deteksi wajah berbasis facial landmark. Jurnal Teknologi Informasi, 5(2), 144–148.
Sitorus, H. (2022). Implementasi deep learning mendeteksi pengguna masker berbasis framework TensorFlow dengan metode convolutional neural network. SENTRI: Jurnal Riset Ilmiah, 1(3), 862–874. https://doi.org/10.55681/sentri.v1i3.298
Wati, R., & Ernawati, S. (2021). Analisis sentimen persepsi publik mengenai PPKM pada Twitter berbasis SVM menggunakan Python. Jurnal Teknik Informatika UNIKA Santo Thomas, 6, 240–247. https://doi.org/10.54367/jtiust.v6i2.1465
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Jurnal ilmiah Sistem Informasi dan Ilmu Komputer
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
