Interaction and spatial compatibility effects of visual and auditory signals: implications for human machine interaction design

Abstract

In any human-machine systems, there is a need of proper integration of human factors along with hardware, software, facilities and other elements in system design. Visual and auditory signals are two commonly used means of communication in man machine interfaces in control consoles. However, relative little attention has been paid to the interaction of stimuli of different modalities in spatial relationship in human machine communication. In the present study, three experiments were conducted to investigate the relationship of visual and auditory stimuli in the context of spatial stimulus-response (S-R) compatibility. The first experiment was designed to examine the auditory S-R compatibility effects in both transverse and longitudinal orientations under the diagonal configuration of signals. The results demonstrated a significantly strong transverse compatibility effect but a weak longitudinal compatibility effect. Besides, it was noted that subjects are more responsive to front-right stimuli, arising from the intrinsic structure of human ear and the right ear advantage for right-handers used in the experiment. In the second experiment, the interaction effects of synchronous and asynchronous presentations of visual and auditory signals were examined. The degree of sensory dominance effect was studied at different time delay intervals and sound intensity levels. The results showed that synchronous stimuli presentation yielded better performance in both response speed and accuracy than asynchronous presentation. Moreover, the visual stimuli tended to dominate awareness of auditory stimuli in synchronous condition and the visual dominance effect existed even when the auditory stimulus was presented 200 ms before the visual stimulus. Nevertheless, the visual dominance effect was found to be weakened to a certain extent with an increase in sound intensity level. Experiment three examined the spatial S-R compatibility effects of visual and auditory signals in the longitudinal and transverse orientations. The spatial compatibility effect was found in this experiment and it indicated that the compatibility effect could be formed with the existence of two different stimulus modalities. Furthermore, it was observed that humans are more responsive to visual signals than auditory ones. The visual strength effect is believed to come into play whenever the subjects have to face the occurrence of visual and auditory signals. This study provides very useful results for establishing ergonomics recommendation guidance in display-control consoles with which the overall efficiency, reliability and safety of the human-machine systems will be enhanced.