Hand gesture game - Gesture Summoner

Abstract

Augmented Reality (AR) has long been a hot topic under the rapid development of Human-Computer Interaction (HCI). Besides traditional keyboard and mouse interaction with computers, huge research efforts have been put to achieve a more natural and human-centered way for us to interact with computers. With advanced technology, we are now having a brand new method to communicate with computers by using our bodies directly like gestures and movement. Besides, new robots with superior computer vision technology can also understand more about the world outside its processor and chips. AR is very important in human life as it not only superimposes virtual object to our real world, it also brings the virtual world into our daily live so that we can enjoy our new mixed reality today. First, we can enjoy different effects brought by digital world which our real environment cannot provide. Second, we can get more information at a time by the enhancement brought by AR. Last, we can control the computer without keyboard, mouse or touchscreen. It is particularly true for gesture recognition as gesture is also a very important part in human communication. Despite the fact that much efforts have been put into the research area, it is still very difficult to detect a hand gesture as it does not have clearly defined structure for computers to do detection. In addition, there is no general algorithm to recognize all kinds of hand. The aim of this project is to study how the computer can understand the real world and recognize a hand gesture. Although there are already many existing studies on detecting and recognizing hand gestures, there is still much room for improvement. This is because some of them are too simple and do not provide enough information for gesture recognition, while some are too complicated that may not be good for real-time AR application. As the computer can only make sense of the real world purely in terms of data, the main objective of this project is to "teach" the computer how to interpret those data into meaningful points, and then try to place images onto those points to produce virtual objects. After that, the result of the study will be used to make an application which allows users to use specific hand gesture to control virtual objects.