An iPhone application with nearest obstacle detection for the visually impaired

Abstract

Nowadays, mobility is still one of the most concerned problems for visually impaired persons (VIP), especially for the totally blind. An iOS application to detect the obstacles surrounding them is developed to aid them in the mobility aspect. The application is written in Swift, with the main function to detect the nearest obstacle in real-time. The detection makes use of the combination of streaming depth data from the device cameras and the YOLOv3 model for object detection to give detailed information about the obstacle in real-time, including the distance between the phone and the object and the direction of the object. The information is converted to speech notification with vibration in a VIP-friendly format. Apart from the main function, the application also supports downstairs detection for the users, which is a function suggested by VIP. In addition, the user interface is designed to be as simple as possible, with only a rectangular view to capture the obstacles and some labels showing the necessary information. A user experience survey has been done, the feedback collected is used in the development and improvement of the application.