Model Based Document Classification and Clustering
Alejandro Murua, Werner Stuetzle, Jeremy M. Tantrum et al. · International journal of tomography and simulation · 2008
In this paper we develop a complete methodology for document classification and clustering. We start by investigating how the choice of document features influences the performance of a document classifier and then use our findings to develop a clustering method suitable for document collections. From our study of the effect of frequency transformation, term weighting and dimensionality reduction through principal components analysis on the performance of a simple K-nearest-neighbors classifier, we conclude that: (a) applying a logarithm or square-root transformation to the term frequencies reduces error rates; (b) term weighting of the transformed frequencies does not appear to help much; and (c) increasing the feature space dimension beyond 50 does not improve performance. We use these findings in the construction of a Gaussian Mixture Document Clustering (GMDC) algorithm. This algorithm models the data as a sample from a Gaussian mixture. The model is used to build clusters based on the likelihood of the data, and to classify documents according to Bayes rule. One main advantage of our approach is the ability to automatically select the number of clusters present in the document collection. Our experiments with the Topic Detection and Tracking Corpus demonstrates the ability of GMDC to choose a sensible number of clusters and to generate meaningful partitions of the data.
Cited by 6 works.
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis
- Fundamentals of speech recognition
- Using Linear Algebra for Intelligent Information Retrieval
- Bayesian Model Selection in Social Research
- Model-Based Gaussian and Non-Gaussian Clustering
- Improving the retrieval of information from external sources
- A Method for Comparing Two Hierarchical Clusterings
- Matrices, Vector Spaces, and Information Retrieval
- Gaussian parsimonious clustering models