Online sketched symbol recognition

Abstract

Sketching is a natural way of externalizing ideas and turning internal thoughts public. People usually use sketches to express and record their ideas in many domains, including mechanical engineering, software design, information architecture and map schematizing. Sketches are acquired initially in the format of point chains. This format requires larger storage but contains less semantics. In order to solve these problems, this thesis presents a novel preprocessing algorithm and a novel syntactical graphic object recognition algorithm to relate a free-hand sketched object with its regularized form. The preprocessing algorithm approximates a stroke by primitives (such as elliptical arcs and straight lines) by extending the dynamic programming framework with a customizable penalty function. With the help of the penalty function, the preprocessing algorithm automatically analyzes the number of primitives contained in the stroke and segments the stroke into several primitives that are consistent with human’s perception. After preprocessing, a stroke is presented in several primitives, whose starting points and ending points are regarded as splitting points. Our syntactical algorithm relates the sketched object to a model object based on the generated primitives. Different from existing syntactical graphic symbol recognition approaches, most of which calculate the geometric relations between primitives and use the relations as constraints for matching, our algorithm builds up a mathematical model to evaluate the geometric information of a primitive with respect to the whole object. The mathematical model, which is theoretically invariant to scaling and rotation, is used to establish a one-to-one mapping between the primitives contained in the sketched object with those contained in a model object. The mapping is then used to measure the similarity between the sketched graphic symbol and the model object. The main contributions of the thesis are in two aspects. The first one is the proposed preprocessing algorithm, which adopts the techniques used in locally optimal approaches into the DP framework, which is the framework commonly used in globally optimal approaches. Compared with existing locally optimal algorithms, our method gives a more accurate result while maintaining the computation efficiency. Compared with existing globally optimal algorithms, our method reduces the computation complexity so that it can be applied to online applications. Our preprocessing algorithm merges the advantages of both locally and globally optimal algorithms while gets ride of their disadvantages. The second contribution is the proposed syntactical graphic symbol recognition method, which measures the geometric information of a primitive with respect to the whole symbol rather than its relations with every other primitive in the symbol to avoid the graphic isomorphism problem, which occurs in almost all of the syntactical pattern recognition approaches and has been proven to be NP-Complete. Experiments show that our proposed preprocessing algorithm is robust to noises and applies to sketches of various sizes. Its response time is sufficiently quick for online applications and its accuracy is sufficiently high for the proposed syntactical graphic symbol recognition. Experiments also demonstrate the significant improvements our proposed algorithm has made to the field of vector based symbol recognition.