Research and development of semantic web application with intelligent web crawler

Abstract

Semantic Web, a state-of-the-art concept of web architecture proposed by W3C, means to change the current web into a machine processable web. This project aims to investigate the values and new capabilities of Semantic Web. The short-comings of traditional web technologies, as well as the literatures of Semantic Web technology are studied. To realize its capabilities, a major part of the project is to work out a Semantic Web prototype and carrying out experiments on this prototype. The prototype built in this project automates web information processing and provides intelligent advice to students in selecting the most appropriate universities and subjects to apply to. Without Semantic Web technology, students need to manually browse through pages and pages of information in Web sites of different Universities to find a list of universities, departments and programs that they qualify to apply for. Our prototype demonstrates that Semantic Web can automate and streamline this whole process. All the students need to do is enter their qualifications and a list will be generated automatically. The prototype uses a shared ontology based on RDF and DAML+OIL to describe admissions requirements of universities in Hong Kong. One of the objectives of this research project is to design such an ontology and to test its limits by encoding admission requirements for several of our local universities into our prototype. This report also presents a novel "explore by class" search algorithm I developed to optimize search performances in this university Semantic Web problem domain comparing to traditional web crawling techniques. With the prototype, several experiments are done to testify the capabilities of Semantic Web to provide better adaptations for information processing in the dynamic, open and loosely controlled web environment.