Video-based Evaluation of Guitarist Performance

Abstract

An effective method to evaluate the performance of an individual instrument artist is a critical component in the instrumental education domain. The existing rating system mostly relays on the subjective opinions of experienced artists instead of a standard scalar assessment. In the paper, a Scalar Performance Evaluation System (SPES) which measures the guitarist performance based on 2D hand gesture recognition using deep learning model is proposed. The key points of the left-hand (chord-hand), which is extracted from a 2D video, without depth image, is analyzed and classified by a Recurrent Neural Network (RNN) into three levels of proficiency. Currently, the model achieves an 85% classification precision for the dataset prepared by the author. This project proves that a noticeable relationship between the hand gesture and the guitarist performance can be observed, and it shows that the Scalar Performance Evaluation System (SPES) is a useful, practical, valid measurement on artist performance objectively. Further research can focus on the extension of the range of detection targets to other instruments and the improvement of the classification accuracy of the system.