Self-Ordering Kiosk with Gesture Recognition

Abstract

During the COVID-19 pandemic, many people raise the awareness of hygiene. It is very easy to spread diseases and viruses through touching. However, when using the self-ordering kiosk in a restaurant, users need to touch the screen or buttons in order to control and use it. The users who did not clean their hands after touching the screen or buttons may be infected by the bacteria and viruses on the screen. There are approaches to reduce the risk of spreading diseases, including sterilization, using food ordering mobile app and self-ordering kiosk that detects voice commands. But they have different disadvantages. There is also exists self-ordering kiosks that detects mid-air hand position. Although it is good in overall, it has some limitations and improvement is needed. This project aims to develop a self-ordering kiosk system which is controlled by using simple static gesture commands and the improved method mentioned. The system uses MediaPipe Hand and neural network for gesture recognition. At last, usability testing will be conducted to collect feedback for the system.