High speed string matching for virus scanning - Quick Sampling and Verification

Abstract

This report introduces the lookup table constructions for the QSV and AU modules of a string searching engine of virus detection purpose. The QSV (quick sampling and verification) method is based on quick sampling of the input data with fixed-length prefixes, and on-demand verification of variable-length suffixes. Patterns are required to possess distinct 16-byte prefix, which is ensured by the segmentation process. Some short and exceptional cases patterns are transferred to pipelined Aho-Corasick (P-AC) module.[1] The AU (aggregation unit) is responsible for concatenating partial segment matches to produce the final pattern match. Only 3 lookup table entries in QSV are occupied for a pattern with unique 16-byte prefix and the overall memory cost is 1.4Mbyte. The lookup tables for QSV are T1, T2, and T3, while the lookup tables for AU are A0 and A1. Several methods and algorithms are introduced to improve the memory efficiencies of the lookup tables by reducing the total memory cost.