Develop a Motion Sensing Game for the Visually Impaired

Abstract

This report will be discussing my FYP of building a motion sensing game for the blind. The idea was initiated by Professor Kelvin. I decided to do the game because while there are numerous motion devices in the market, the target customer is usually the developer or the more adventurous gamer. When I did my research for audio games for the blind, most of them are either outdated or not developed enough to make the game challenging or interesting. I believe that the blind is an untouched target demographic for interesting games. The motion sensing game has been developed with Unity, IBM Watson and the Myo armband. The player will be given the task of finding the audio source around the game field by moving left to right. Unity was used to build the game, the Myo armband has been used to do the motion sensing part (gesture detecting) of the game, and IBM Watson has been used to incorporate text-to-speech elements of the game. All voice instructions/command system of the game was built using IBM Watson. Despite my inexperience with all the platforms (Unity, Myo and IBM Watson) and programming languages (C#) in the beginning of the project, I have learnt that the visual limitations made the development process more challenging and interesting. I learnt a lot about how to develop 3D sound, playing with the armband's accelerator, and using cloud technology for the audio instructions. If I were to develop this game further, I would most definitely integrate machine learning to study the user's behaviour and patterns (data mining). Another option is to integrate multiplayer game system so that two or more visual impaired people can play the game.