Indoor Scene and Object Recognition System for the Visually Impaired People (Mobile Application)

Abstract

One of the major limitations to the visually impaired people is the incapability to quickly recognize the object and the scene in the surrounding environment. This reduced the mobility in indoor environment and it is very hard for them to search for objects such as cup and bag. In this project, two versions of Android application to realize the surrounding object and scene are proposed with the aim to improve their mobility and facilitate indoor object access. For version 1, an Android application is developed using You Only Look Once v3 - tiny (YOLOv3-tiny) trained on COCO dataset as object detection method with SVM trained on the object features from MIT Indoor 67 as scene recognition method. For version 2, an Android application is designed to combine YOLOv3-tiny object detection model trained on ImageNet + COCO datasets and MobileNetV2 scene recognition trained on MIT Indoor 67 using TensorFlow Lite. Version 2 is still in progress. Since limitations exist with the method use in version 1 for scene recognition, version 2 is proposed with the aim to improve the scene recognition accuracy. The methods used in this project are lightweight neural network dedicated for real-time implementation. The user can gain better understanding to the surrounding by using the application on an Android mobile with a camera. The interaction with the application is done by voice input and output on the mobile.