Automatic Posture Keypoints Detection using Convolutional Neural Networks - IPS CityU Apps Lab

Abstract

This project serves to improve upon the CityU Apps Lab's Posture Check mobile app by proposing a pipeline for the automatic detection of posture keypoints in the full-body front and side view images of human users. In the method proposed the keypoint estimation is formulated as a regression problem which is solved using a deep learning approach. The first half of this project is concerned with the development of a lightweight Convolutional Neural Network (CNN) architecture for human pose estimation on the FashionPose dataset. The model is shown have comparable results with the current state-of-the-art. In the latter half, a new dataset of 900 images annotated with posture keypoints is used to adapt the pre-trained CNN to the new domain of posture keypoint estimation using transfer learning techniques. Two approaches to transfer learning are explored and evaluated. Finally the quantitative and qualitative results from the completed pipeline are presented. The final results of the pipeline demonstrate its high detection rate which it achieves while maintaining a fast prediction speed and comparatively small memory footprint.