Natural language understanding for query in web search – 1

Abstract

In this project, a search engine is developed for searching the nutritional information in a more effective manner. It is written in Microsoft .NET environment with C# language, and based on a nutrition lexicon database. To obtain better search results, natural language is applied in processing the web query. To implement a search engine, it involves several processes: querying, crawling, information retrieval and ranking. Natural language processing is applied in the querying process by expanding the query entered by users. Google API is used in the crawling process to access the web document database of Google. Vector space model is applied in the information retrieval process with three different approaches: 1) Static query 2) Dynamic query 3) Dynamic query with relevance feedback. By using vector space model, a weight is then calculated for each retrieved document and these weights are eventually used to reprioritize the documents in the ranking process. In order to select the best approach, a software ‘SPSS’ is used to evaluate the search results. Among the three approaches, the third one is time inefficient with Google API timeout problem whereas the others obtain a better retrieval performance when compared with Google’s search results.