Visual speech for recognition system

Abstract

When performing speech recognition, we have to face the problem of noise around. The lip reading system is mainly a support for speech recognition. In this project, a system is developed to extract the user lip information from an image grabber, and use such information to provide a prediction of possible words that the user speak. In order to extract the required lips information, several parts are implemented to achieve the target. They are Lip Position location, Lip Clustering, and Recognition. The Lip Position initialization is used to find out the lip position from a grabbed image. It can help to reduce the complexity of Lip Clustering. After that, image is segment into lips and non-lips color. The method is Fuzzy clustering. The image is then fitted in a lips model to find the contour. Finally, it is the part of recognition. The Hidden Markov Model (HMM) was used to train and perform the recognition. Some parameter from the extracted contour is chosen for training the model. The result shows accurate recognition on words with special lips shape change. But for the words with similar lips movement will give wrongly recognition.