Developing a multisensory feedback system in immersive virtual reality for construction safety education

Abstract

Thanks to the advances in engineering technologies, the emerging Virtual Reality (VR) technology has been adapted as a novel and alternative tool to improve safety training, featuring the immersive and vividly training experience to users. However, most of VR-based safety training systems focus on the visualization of the safety rule contents, but there are few effective interactive methods which can give students effective feedback for their safety performance. In this project, the project team aims to develop a multisensory feedback system in immersive virtual reality safety training to improve construction safety education. First, a VR-based prototype featuring a construction job site in a virtual reality environment will be developed. Based on the occupational safety and health statistics 2019 by the Hong Kong Labour Department, five typical risk events caused by human factors are selected involved in the VR scenarios. Next, human-computer interaction functions in each scenario related to five typical risk events will be developed. The over-lopping methods based on students’ current location and risk area will be applied for determining the violation behavior. A multisensory feedback system containing visual feedback, auditory feedback, and haptic feedback will be integrated into the VR-based safety training system, aiming to provide multiple complementary feedback when hazards go unrecognized. Current students in the ACE department will be invited to trial the prototype and provide data through the experiment. The student volunteers will be randomly divided into two experimental groups and freely explore each VR-based training system(system with and without feedback). Right after the experiment, the students will be asked to take a paper-based questionnaire based on the Positive and Negative Affect Schedule (PANAS) scales and the Technology Acceptance Model (TAM) questionnaire. Students’ activate emotional arousal and satisfaction towards VR-based technology will be assessed by the PANAS scales and TAM questionnaire, respectively. To get insights into the effectiveness of the multisensory feedback system in immersive virtual reality safety training, the comparable results for each VR-based training system will be collected and analyzed. If successful, the proposed multisensory feedback system could also be extended to other VR-based methods to help them better evaluate the students’ performance.