Examination Paper Question Analysis

Abstract

DSE is a public examination for students to get into university. It plays a vital role in determining someone's career path and study path. Therefore, this project aims to analyze the most commonly examined topics for future public examination. By revealing the knowledge points and frequently examined formats, it is hoped that students can revise in a more efficient and effective way. There are total 30 years of Mathematics pastpapers to be analyzed. Throughout the project, three stages are involved, which is text mining, classifier building for topic classification and analysis, and question/topic prediction. Python is used for implementing the machine learning algorithm as it contains many essential packages on building the project. For the first two stages of project, Bag of Words model is introduced. Other techniques, for example, preprocessing the documents by removing stopwods and stemming; standardizing the mathematic equation format to increase the classifier's understanding of words; finding the relatively important keywords and knowledge points of each topic; customizing token patterns to leave distinguishable features in feature vector. To evaluate the system, two classifiers (logistic regression and multinomial Naïve Bayes) are employed with various parameters setting of distinct vectorizers. For the last stage, 2018 HKDSE mathematics paper is predicted. It reveals commonly examined question structure. Not just the implementation and result, limitations and future improvements of this project are also discussed.