Development of an augmented reality human-machine interface for promoting student's engagement in robotics and control related courses

Abstract

Wearable augmented reality (AR) devices have become increasingly popular and are believed to transform the future technology of human-machine interface (HMI). Our teaching activities need to prepare our engineering students for the trend of immersive HMI technology in both the lectures and hands-on lab experiments. This teaching project is developed to improve current robotics and control related courses such as MNE3058 Embedded Control Systems, MNE4032 Robotics and Machine Vision. Although the hand gesture, voice recognition, and 3D visual display are introduced in the courses as essential techniques for HMI, students are currently confined in classroom lectures and only imagine how those techniques are used for the robotic and control systems. According to past teaching experiences and students' feedback, there is a significant lack of interaction and practising opportunities. In this project, we aim to improve the students' engagement by developing a new immersive learning environment with an AR device (i.e., Microsoft HoloLens). Multiple HMI modules will be programmed to allow the students to interact with the AGV robot easily. Additionally, wireless communication programming will be included as another excellent lab experiment for the student to learn embedded communication. With a rapidly growing market, the 3D AR technology may soon become universal learning tools. This TSG project is also a pilot study to investigate the broad adoption of AR in teaching engineering laboratories.