Sketching and sampling techniques for colored range queries

Abstract

The large amount of memory required by storing information is always a problem of many data administrator. Sometimes, data points are associated with categorical attributes and queries are interested in information about the categories of the points. We used colors to represent the categorical information. A colored range query asks for the information of the colors of points lying within a query region. To reduce storage and improve query response time, we apply synopsis data structure to summarize a very large colored vector to answer the colored range queries. This project will show the feasibility of answering colored range queries by synopsis data structures and discuss various implementation problems and solutions. L0 sketch and coordinated sample will be selected as the synopsis data structure for colored range count query in this project. We first test their performance on estimating the number of distinct non-zero elements in a vector; this is the original usage of them. Then we will evaluate their performances on answering the range queries in terms of approximate accuracy, memory usage and speed to compare which synopsis data structure is better. We also tried some variations on the L0 sketch to measure the effects of different setting.