QProber
Luis Gravano, Panagiotis G. Ipeirotis, Mehran Sahami · ACM Transactions on Information Systems · 2003
The contents of many valuable Web-accessible databases are only available through search interfaces and are hence invisible to traditional Web "crawlers." Recently, commercial Web sites have started to manually organize Web-accessible databases into Yahoo!-like hierarchical classification schemes. Here we introduce QProber, a modular system that automates this classification process by using a small number of query probes, generated by document classifiers. QProber can use a variety of types of classifiers to generate the probes. To classify a database, QProber does not retrieve or inspect any documents or pages from the database, but rather just exploits the number of matches that each query probe generates at the database in question. We have conducted an extensive experimental evaluation of QProber over collections of real documents, experimenting with different types of document classifiers and retrieval models. We have also tested our system with over one hundred Web-accessible databases. Our experiments show that our system has low overhead and achieves high classification accuracy across a variety of databases.
Cited by 135 works. doi:10.1145/635484.635485
Authors
References
- Pattern classification and scene analysis
- C4.5: Programs for Machine Learning
- Indexing by latent semantic analysis
- Introduction to Modern Information Retrieval
- A Tutorial on Support Vector Machines for Pattern Recognition
- Fast Algorithms for Mining Association Rules in Large Databases
- Fast algorithms for mining association rules
- Human behavior and the principle of least effort
- Term-weighting approaches in automatic text retrieval
- Pattern classification and scene analysis